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ABSTRACT 



The purpose of this project was to adapt or develop and to field-test 
an evaluation model for co.npetency-bas«d education, using the FACIT teacher- 
training materials as the focus for model development and using these mat^rl? 
als during the field test of the model. Further efforts were to center on ' 
expanding the model to apply to other competency-based programs. 

Because of problems with* the production- of the FACIT materials, the 
field test of an evaluation model could not be carried out. No attempt 
was made then to develop a model tailored to the specific needs ofFlorlda 
schools; Instead the project sought a model that could be easily adapted 
to the evaluation of any competency-based prdgram. . 

To meet this revised goal, an In-depth literature review of evaluation 
mqdels for competency- based programs was undertaken, using a set of criteria ' 
that would satisfy the requirements of program description and evaluation. 
The^purpose of this review was to determine whether a model for evaluation 
of competency-based programs existed, one that could be either adopted or 
modified to meet the established criteria. Since the literature review on 
evaluation of competency-based programs failed to yield any models that 
were sufficiently comprehensive, the literature review was expanded to 
Include general evaluation models that were (with some modifications) also 
applicable to the evaluation of competency-based programs and that met the 
established criteria. 

As a result of this extensive review, the Stufflebeam model was Identi- 
fied as one that would provide comprehensive guidelines for planning the . 
evaluation of- competency-based programs. This model, however, did not pro- 
vide sufficient, guidance In the description ov programs to be evaluated. 
Therefore, the project developed program-description guidelines that Incor- 
porated elements of the Stake evaluation model and th^ Concerns-Based 
Adoption Model . . 



These guidelines for describing programs to be evaluated recommend a 
two-phase process. Phase One Includes program description for the purpose 
of planning the evaluation, and Phase Two Includes behavioral de&c* iptlons 
for the purpose of making evaluative judgments. / 



It Is recommended In this report (a) that evaluators use these guide- 
lines but develop their own procedures and use Instruments to meet the 
Information requirements of any given situation, (b) that FACIT evaluators 
become thoroughly familiar with FACIT and modify CBAM techniques to meet . 
FACIT' evaluation Information requirements, and (c) that evaluators who 
use these guldlDlnes, whith have not yet been field-tested, document their 
experience and suggest further developments. 

Appendices to this report provide the evnluatorswlth detailed Infor- 
mation about evaluation questions and with Instruments that can bo used 
In evaluating competency-based programs. 




Introduction 



The purpose of the project was to adapt or develop an evaluation model that 
could be used, to guida evalu'ation of the FACIT (an acronym for Florida's Approach 
to Competency-Based Individualized Teaching) teacher-training materials. The 
evaluation rtiodel was to be adapted or developed and field-tested in conjunction 
with the field-testing o^ the FACIT materials. In addition t6 its use in evaluat- 
ing FACIT, the model would serve as a more general framework for evaluating any 
competency-based program, regardless of content, educational level, or institu- 
tional setting. To enhance the genera 1 i zabi 1 ity of the model, plans were made to 
address the needs, of a variety of educational, decision-makers, including teachers; 
'school administrators; and district, regional, and state-level administrative 
personntl. 

In adapting qr developing the evaluation model, the following specific objec- 
tives nere to be accomplished: 

1. To determine specific evaluation questi(^ns to be answered in each 
field-testing site 

2. To determine the' information needed to answer the evaluation questions 

3. To determine sources for obtaining the evaluation information 

4. To determine procedures for collecting and analyzing evaluation information 

5. To determine evaluation decisions and cons'traints on making these decisions 

6. To test the evaluation model on FACIT field-testing sites 

Unfortunately, the above objectives could not be achieved because of problems 
encountered in the production of the FACIT materials. Although considerable staff 
time was dpvoted to, attempts to resolve the problems with the graphic design stu- 
dio holding the contract to produce the materials, finished materials were not 
available. The project's objectives were, therefore, mddified to eliminate the 
collection of specific evaluative information on FACIT and on the specifics of 
carrying oiit state Or local evaluations of competency-based vocational programs. 

Before deciding whether to adapt an existing model or to develop a new one, 
the project staff searched the literature on educational evaluation to determine 
what was available with respect to the evaluation of competency-based education. 
A set of meta-evaluation criteria was used to determine the appropriateness of 
existing models for the evaluation of FACIT and of other competency-based voca- 
tional programs. When little that related to the evaluation of competency-based 
programs was found, the search vas expanded to encompass educational evaluation 
in general. 

After the most relevant evaluation models were examined in some detail, it 
was decided that the evaluation procedures recommended jy Stufflebeam, et al.,* 
provided a comprehensive guide to .the conduct of program evaluation for competency- 
based education, but that more guidance in program description was needed. Plans 
were made, therefore, to supplement the Stufflebeam model with program-description 



'Stufflebeam, D. L., et al. Educational evaluation and de cision making . 
Itasca, 111.: F. E. Pea,cock, 19/1. 
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guidelines. The best elements of another well-known evaluation model^ and of 
a research-based model of the process of facilitating the adoption of education 
Innovations^ were combined to yield the program-description guidelines necessary 
to successful evaluation of educational programs. 

Metf^ods 

TH.1S section focuses on the criteria and decisions that led to the program- 
description guidelines presented In the Results section. First, 'the criteria 
used to guide the literature search are presented. Then, the decisions Involved 
In the various phases of the literature search and pf the development of the 
program-descrlptl'On guidelines are described.. 

Before the decision was made to focus on program description, the project 
attempted to locate a model that could b^ adopted or adapted for evaluating 
competency-baseti vocational programs. To assist the search for a satisfactory 
model, criteria reflecting concerns common both to program evaluation in general 
and to evaluation of competency- based education in particular were developed. 
The following is a list of the criteria. 

1. The evalAjation modef should guide the evaOuator in asking questlons* 

that are specifically relevant to competency-based vocational education- 
questions such as: -v- . 

a. Are the program's goals or competencies based on empirically 

identified needs of the school's service area? 
» b. Are students able to perform on the job those skills that they 

have mastered in the program? 
c. How will program costs change when the whole program is individualized? 

2. The evaluation model should guide the evaluator in adequately describing 
the program and the context in which it is Implemented. Guidelines should 
be included for identifying or describing the following: 

a. The needs the program is supposed to meet 

b. The intended and actual physical, social, or psychological factors 
(both positive and negative) that affect the planning or implementa- 
tion of the program • 

c. The rationale and assumptions underlying the program 

d. The goals and objectives of the progr.ir 
• e. The actual outcomes of the program, inclTN^ing those not planned for, 

whether positive or negative 
f. The processes, procedures, etc. , that wei e |1ntended or actually 
Implemented in the program 



^Stake, R. E. The countenance model of educational evaluation. Teachers 
Colleg e Record , 1967, 68, 523-54- 

'Hall, G. E., et al. Levels of use of an innovation:. A framework for 
analyzing innovation adoption. Journal of Teacher Education. 1975, 26(1), 52-56. 
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3. The evaluation model should provide Tiexible guldellnels for conducting 
the evaluation. 

a. A variety of data-collection Instruments should be reconnended , along 
with suggestloqs for their use. 

b. A variety of data sources should be reconnended. 

c. Recommended procedures for collecting, organizing, and analysing 
Informatlorr should be clearly related to the evaluation questions. 

d. Guidelines should be Included to facilitate the Identification of 
appropriate decision makers. 

e. Guidelines should be Included to facilitate decision making. Including 
the establishment of standards and the assignment of weights to factors 
that mighf Influence the decisions. 

4. The evaluation model should provide guidelines for reporting evaluation 
results In a way that can be tailored to the needs of a diverse target 
audience. 

5. The evaluation mood should provide guidelines for designing evaluation 
plans that minimize costs and_.the use of outside resources In their 
Implementation. 

The above criteria, which are rather general, served mainly to generate spe- 
cific questions about the evaluation models reviewed. Tables 1 arid 2 are further 
examples of the kinds of general and specific criteria that helped In the search 
for a model for evaluating competency-based vocational education programs. 

Th'' search for such a model was conducted In two phases. In the first phase, 
the search was restricted to models specific to the evaluation of competency-based 
programs. In the second phase, an examination of the more general evaluation 
literature took place. ^ 

Although much has been written about competency-based. education In recent 
years, there Is little In the way of guideline: for conducting program evaluations. 
Much of what has been written about evaluation In competency-based education has 
dealt with the evaluation of student performance, rather than with the larger 
Issue of program evaluation. (Note that evaluating student performance Is an 
Important element pf. program evaluation, but It Is certainly not the oniy element.) 

g 

The first phase of the literature se«Tch yielded no models that adequately met 
the criteria listed on pages 3 and 4 and In Tables 1 and 2. There were some arti- 
cles,** however, that dealt with Issues that are Important In program description. 
Relevant approaches and Ideas In these articles were subsequently Incorporated 
Into the program-description guidelines; the decision to do so, however, was not 
made until after the search of the more general evaluation literature yielded a 
suitable model for conducting program evaluations. 



''Andreyka, R., & Blank, B. A checklist f{|| the evaluation of competency-based 
teacher education programs. Educational Technology , January 1*J76, 34-37. 

Denton, J. J. A field- tested evaluation model to assess a CBTE program. 
Educational Technology , March 1977, 17, 23-27. 

Hirst, B. A., Jr. The components of competency-based vocational education. 
American Vocational Journal, November 1977, 52, 32-35. 
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TABLE 1 • 

' <• * KVALI ATIDN lU VltW CHIXKUST:* 

. For Review of Deslfn. and Report, of Kvalu.tion studies on Instructional Products and Systems 

Context Desc'rlotlone (Overall) ' • . 

A. Peocriptfon of Product/Sy tem (Overall) 

ingT„;riv';^!::;?.,°:r^e:ir ' »' c«n;e„;. 

J. . I. p.,™i«i„„ „, cio.ri,".p;cifi«i-. Ba.ic d,!.;,;,. i„;om;„.,w p^o'dur;-, '^.tiin^, : : : : : 

B. Purpose of Evaluation (Overall) 
1. I;;ocu8 -answers wliat in being evaluated and why > 

gvaluatton Activities/Resultg (Overall) 

\* Evaluation Qui*8tiona (Overall) = • 

Data Sour cea/Collectton (bve ral I) 



^ 5 



.'i^JJl^ v. Manniit 1' cnt C onc orngi (Over all) 



Cr Data Proccssint^/Heportin j ^ (Overall^ 



clear and warranted'. 
^ivune. - no,K,rt.ng audiences specified? Reporting techniques* approirlaie^ 



'^'Iv fi^UimV.':M■^•''■il£l«•r!^;.l^H (Overall) ; , . . . 

A. .EM11!!1^«_ - if tlu. i,m^,nHv of tlu- .I,.cument clear? ^Are the informntii 
jt ». /V(i(inm.»' . jiiv Ihf iltiLMimniit'H iiit<....ii..< !»,- . . 



V. >J::;L^j>y.L'!:?2 - iiK- »KK-umi>nt c.lm.. u, ihr point? Tel. t'-c o«sr-i ,U « m,l. . ■ v ^ 

8„-^urce: AE«A-5pon.ored workshop cooductod by B. Wor't^ea ^oJ o:'u^;ZZ:m^^:'?\^7Wi: 

aj^ana •This cIuvUUm .Ir.mi. laMvily on work by d riven, Stoke, Murray mul :*mi:h 



' TABLE 2 ■ 

€ons1 derations for Evaluati.on>lan: ? 

Conce ptual Clarjty . • * , " * * . 

t 

t Is evaluation formative or summative? 

* I.S evaluat1,gn comparative ^or single program? * 

* Is evaluation goal-directed 'Or'goal .fjree?. 

'2. Characterization of the Object of the Evaluation ^ 

* Is there a complete and detailed description of the program? 

3. Recognltfon and Representation of Legitimate Ai^ilences 

« 

* Is there Input from and reporting to all audiences? 

* Who Is the aujllence for the evaluation? 

4. Sensitivity to Political Problems * ' ' ' 

. * ■ * ■ 

* Vlhat are the constraints on data collection? 

ft 

5. Information Needs and Sources * 

* - What are they? ' 

* How do they relate to evaluation questions? 

^ . .. 
•6. Comprehensiveness and Incluslveness 

Have data related to objectives been collected? 

* Have data on side effects been collected? 

c 

7. Technical Adequacy 

* Are sampling, measurement, and design concerns accounted for? 

8. Cost 

* Is the plan costreffectlve? 

■4 

9. Standards/Criteria - ^ 

9 

* What are they? " . 

10. ' Judgments and/or Recommendations 

* Is th»» program effective? , 

.* What recommendations can be listed? 

11. Reports Tailored to Audiences 

* Do you need technical reports? . 

* Do you need nontechnical reports? 

* Win feedback on Interim reports be analyzed? 

Source; Unknown 



* ThC'Same criteria used to examine. the competency-based literature' wi the 
first phase were ,used in the second! phase of the literature search. ^This search 
of the more general evaluationjitera'ture led to a detailed consideration of and 
su|)$equent adoption of the St;o'ff1et)eam' guidelines for conducting fxrogram evalua- 
tion (see-Jable 3 for an outline of the evaluation tasics li^sed on.Stufflebeam's 
model), these guidelines seemed sufficiently comprehensive (according to the 
•criteria listed on pages 3 and 4* and ih tables 1 andv2i an<) fl.exible to^guide 
the evaluation. of any 'competency-based program. » ' * » 

While the Stufflpb^am model was the most comprehensive one found,* it lacked 
si4ffi&ient guidance f6r describing the program to be 'evaluated. Thus, t)re search 
'continued' until it. was clear that no additijonal? information>»uidL be gained. On 
the basis of t|)e literature review, it was decided to incoi^grate, into the pro- 
gram-description guidelines elements of tl) Robert Stake's model for describing 
and judging educational program? anc :f (2), the Concerns-Based "Adoption Model-. 

Stake's mcdel wa5 chosen becausft'it provides evaluators with a general frame- 
work <or'<lesCrjbtng- educational programs.. Using Stake's model; the evaluator is 
directed to loolc not only at program outcomes, but also* at program processes, 
antecedent conditions; and the contingencies that exist, between thes/*^ three 
categories of information. In addition, the evaluator is directed io Itwk at 
the congruence betweefj whfiit was intended in relation to. the program and what 
actually took place. -Another important selling point for the Stake model* i!i 
that s.tandards and the judgments based on those standards are related, 'In a one-" 
to-one fashion, to the categories of information in the program description. 
Furthermore, Stake's model allows both absolute as wall as comparative judgment's, 
a feature that increases its adaptability. (See Figure. 1 on page 8 for a griphlc 
illustriition of Stake's model.) , 

Stake's model, as comprehensive as it is, does not provide .sufficiently 
detailed guidelines for the collection of description information about the 
program or its implementation. The categories of information are supplied, but 
the details of what to look for are not. The^efore, the guidelines for describ- 
ing programs (to be presented in the Results section), while incorporating the 
information categories from the Stake model, also include. elements from the 
Concerns-Based Adoption Model . . - » s 

AUhough*not intended/to be an evaluation model, the Concerns-Based Adoption * 
Model has implications for .those involved in evaluating educational programs.. The 
model, graphically depicted in Figure 2 (page 9). views the adoption of ?n educational 

innovation as a process that takes place over time and that varies from p^rsor\ to' 
person depending on individual needs, concerns, capabilities, and so on. Two ' , ^ 
concepts from the model and their related instruirtentation have particular signifi- 
cance foif ewluating competency-based vocational progreyns. These are (1) thp 
concept of s tages o f concern about the innovation (see Table 4 on page 11) an^ (2) *»' 
the concept ofieve Ts of u§e of the innovation (see Table 5 on page 12).. 

Specifically, the concepts of stages, of contern and levejs of use help the., 
evaluator to operationally describe the program as it is being implemented. 
Instruments and procedures have been developed to document the concerns of pro- 
gram users and th'e "'e^ttent to which they are implementing the program, "^he model 
also provides for the description of the level of use within various categories 
of implementation. Table 6 on page 13 lists the implementation categories and, 
their definitions . 
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Adapted from-Stake, R. E. The countenance model of educational evaluation. 
Teachers College Record , 1967, 68(7), 523-540. 
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Figure 2 

Components of the Concerns-Based Adoption Model ^ 



USER SYSTEM 

Analysis of 
Institutional Needs 

,Ana.1ys1s of 
Institutional 
Capabilities 

Resolution of 
' Concerns 

Progress In Usage 
arid Readiness for v; 
Use 

Progress In 
Stage of Users' 
Concerns 



LINKAGE ' 
CHANNELS INVOLVING 
CHANGE AGENT 



INFORMATION 
Needs 

Capabilities 

Concerns 

Usage 



ACTION 

Probing for Concerns 
Orientation* 
Training 
Cons ultation , 
Treatment 



RESOURCE SYSTEM 

Innovation 
Development 

Training System 
Development 

Concerns Analysis 

Usage Analysis 

Diagnosis and 
Evaluation 

Action and 
Treatment Selection 



Taken from Hall, G. E.; Wallace, R. C; & Dossett,' W. E. A 
developmental conceptualization of the adoption process within . 
educational institutions. Austin; The Research and DevPlnpmpnt 
Center for Teacher Education, The University of Texas at Austin, 
1973. 



Table 3 

Evaluation Tasks and Activities^ 



I. Delineate Information needs 

A. Define the system to be evaluated 
1. Develop a model of tne system 

a. Set system boundaries 

b. Define elem'ents of the system 

c. Define characteristics of system elements 
D. Specify the decisions to be made 

1. Describe antecedents 

2. State decision setting 

a. State decision authority 

b. State decision responsibility 

c. State decision Influences 

d. State clientele for Information 

e. State decision timing 

f. Summarize decision questions 

3. Establish criterion variables 

a. State questions to be answered 
'• ,b. State alternative answers to questions 
c. State alternative actions 



4. 



State decision rules 

a. Set single-variable decision 
rules 

b. Set multlple-.varlable 
decision rules 

Identify and utilize available 
evidence 
State evaluation policies 
1. State access to data sources 
State access to data base 
and Information 
State role of evaluation 
authority and responsibility 
State budget and resource 
limitations 

State scheduling limitations 
State reporting policies 



5. 



2. 
3. 
4. 




C. State evaluation assumptions 

1. State sampling assumptions 

2. State treatment assumptions 

3. State measurement assumptions 

4. State analysis assumptions 

5. Model the evaluation design 



II. Establish plan fof obtaining Information > 
A. Collect data 

1. State Information source (sample) 

a. - Establish sample size 

b. State sampling procedures 

c. Establish population 

2. State Instrumentation 

a.' Match Items to criterion variables 
b^ — Des^ ib e instrument typ e 1 



c. Specify items of information 
3. Describe collection conditions 

a. Establish responsibility for 
instrument administration 

b. Schtiule instrument administration 

c. Establish setting for administration 
B. Organize data 

1. State unit of organization 

a. Establish level of disaggregation 
required 



b. 
c. 

-dr 



b. Set scoring or coding format 
^Establ ish storage and retrieval 
'requirements 

a. Establish coding format 
for storage 

Document storage procedures 
Specify storage and retrieval 
facilities 

■Spec 1 fy r et r ieval p r ocedures 
3. Establish quality control 
procedures 

a. Establish editing procedures 
^^^r0v4de-€rfK )r ch e c ks 

c. Establish audit trail design 
Analysis of data 

•1. State unit of analysis 

2. State analysis method 

3. State analysis facility 



III. 



Provide information 
A. Prepare reports 



1. 
2. 



Define report audiences 
Depict reporting levels 

a. Specify micro-level reports 

b. Specify macro-level reports 
Describe reporting mode 

a. Establish reporting media 



b. Establish report content 

c. Establish reporting setting 
4. Establish reporting schedule 

B. Disseminate reports 

1. State procedure for trans- 
mitting reports 

2. State procedure for pub- 
lication of reports. 



'Adapted from Scjfflebeam, D. L., et al. Ed ucational eval uat ion and deci si off-making . Itasca, 
111.: F. E. Peacock Publisher*;, Inc., 197T. 

•'(^ 



ERIC 



Table 4 

stages of Concern About the Innovation* 



y AWARENESS t Little concern about or involvement with the innovation is in- 
dicated. . • 

1 INFORMATIONAL ; A general awareness of the innovation and interest in 
learning more detail about it is indicated. The person seems to be un- 
worried about himself. 'herself in relation to the innovation. She/he is 
Interested in, substantive aspects of the innovation in a selfless manner 
such as general characteristics, effects, and requiremei ts for use. 

2 PERSONAL t Individual is uncertain about the demands of the innovation, 
his/her inadequacy to meet those demands, and his/her role with the innova- 
tion. This includes analysis of his/her role in relation to the reward 
structure of the organization, decision making and consideration of poten- 
tial conflicts with existing structures or personal commitment. Financial 
or status irrplications of the program for self and colleagues may also be 
reflected. " 

MANAGEMENT ; Attention i5 focused on the processed and tasks of using the 
Innovation and the best use of information and resources. Issues related 
to efficiency, organizing, managing, schedulinjg, and time deroanda are 
utmost. 

COWSEO^TEN^P ; Attention focuses on impact of the innovation on students in 
his/her immediate sphere of influence. The focus is on relevance of the 
innovation for students, evaltiation of student outcomes, including perform- 
ance and cogpetencies^ and changes needed to increase student o u tcomes. 

5 COLLABORATION ; The focus in on coordination and cooperation with others 
regarding use of the innovation. 

6 REFOCUSING i The focus is "on exploration of more universal benefits from 
the innovation, including the possibility of major changes or replacement 

with a more powerful alternative. Individual has definite ideas about al- 
ternatives to the proposed or exiting form of the innovation. 



4_.:_.. 



"Taken from Rutherford, W. L. An investigation of how teachers' concerns influence 
innovation adoption . Revised version of a paper presented at the annual meeting of 
the American Educational Research Association, Session 9.02, April 5, 1977, in 
New'York. 
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Nnktt n doeition to oto tht innovntion hy tttnb- 
linhing n Urn to bogin. 

•Uto in ifhioh tht nnnr it prtpnring for f imt uto of 
tho innovntion. 



n mfMATIGM 

t] 

Oncition Foint C Chnngtt, if tity, tnd utt tm do^ttd by uttr nttdt. 



ixx mcttutzcALm 



ftnto in which tht nttr foouttt noot tf *ort on tho 
thort-ttrs« dny*to-dny ntt of tht innovntion with 
littlt tiiik for rtfltetion. Oungtt in tiM nrt nndt 
MM to Mtt nttr ntodt thM elitnt nttdt. Tht uttr 
it prinnrily tngngtd in n tttpwitt ntttnpt to nttttr 
tht tntkn rtfuirtd to utt tht innovntion, ofttn 
ntultlng in dinjointod nod tuptrficinl ctt. 



Oocition roint O-l h routinn pnttom of ^no it tttnhlithtd. 



XVh nOUTXMB 



Ott of tht innovnUon it ttnbilittd. rtw« if tny* 
Ohnagtt trt bting nndt in ongoing utt. Uttlt prtp- 
nrntitn or ttought it bting givtn to itfrroving inno- 
vntion not or itt eonttqunnetp. 

Oocition Point 0-2 Chnngtt utt of tho innovntion bnttd on foiMl or 
infoiiitl tvnluntien in ordtr to incrtntt client 



ntntt U which tht nttr vnritt tht utt of tht inny ft; 



tioo to incrtntt tht iapnet on clitntt within tht 
iMtdintt tphtrt of influtnct. Vnrintiont trt btttd 
tn kaowltdgt of both thert* nnd long-ttx# conttqutnctt 
for clitntt. 



Doeition Point I 



Dooition Point P 



Initintot chnngtt in utt of innovntion tonttd on 
input of nnd in coordihntion with wbnt ooUtngunt 
nro doing* 

SUtt in ^ch tht UMr it eonbining cm tffortt to 
MO tht innovntion with rtlnttd netivitict of eol- 
longutt to tohiovt n collective inpect on clitntt 
within thnir comm tphere of influnnM. 

•ogiM enploring nltemntivet to or Mjor nodificf 
titM of the innovntion preeently in mc. 

tthte in which the uter rMvnluntet the qunlity of 
OM of the innovntiM« tttkt mk^ot Mdificntiont of 
or nltcfMtivM to prettnt innovntiM to nchitvt in- 
OTMMd inpnct on client t« exnninet ntw dtvelopMntt 
in tht fitld* nnd tnploret now gonlt tor mU nnd tht 
tyetM. 



ffM the UU Chert > Auetin: ' leeenrch nnd DevelcpOMt Center tor Teneher 
Iducition, The tfnivertitT of TtMt nt Auttin, t97S« 



illcatlons for planned 



evaTiirtlon and leyeUipTl otj III ?th>'<w?>b\%tfon ' ' 

s tudlts . Paper presented at the annual meeting of the American Educational Research 



* Taken from Hall* G. E. Im 
evaluation revealed In the 
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Table 6 



Functional Categories of Use of An Innovation 



1 0 



The categories listed and described below represent typical behaviors of 
the Innovation user. A user's behavior, or level of use (sei; Table 4), may 
vary from one category to another. 



Category 
KNOWLEDGE 



ACQUIRING INFORMATION 



SHARING 



ASSESSING 



PLANNING 



STATUS REPORTING 



PERFORMING 



Description 



i 



Knows about characteristics of the i^rjovatlon, 
how to use It, and consequences of Its user*^ 
This refers to cognlt.ve knowledge related 'to 
using the Innovation, not to feelings t)r 
\att1tudes. » - 

Solicits Information ^bout the Innovation In 
a variety of ways--quest1on1ng resource 
persons, corresponding with resource agencies, 
reviewing printed materials, and making visits 

Discusses the Innovation with others; shares 
plans. Ideas, resources, outcomes, and 
problems related to use of the Innovation. 

Examines the potential or actual use of the 
innovobion or some aspect of it. This can 
Involve simply a mental assessment or the 
actual collection and analysis of data. 

Designs and outlines sh9rt- and/or long-rang»> 
steps to be taken durini process of innovatioii 
Kooption;- i.e., aligns 'resources, schedules 
activities, meets with others to organize 
and/or coordinate use of the innovation. 

Describes persona l posit ion a t the present 



time in relation to use of the innovation.. 

Carries out the actions and activities 
entailed in operational izing the Innovation. 



Adapted from Hall, G. E., et al. Levels of use of the innovation: A framework for 
analyzing innovation adoption. Journal of Teacher Educa tion. 1975, 26(1), 52-56. 



13 



Together, the Stake evaluation model and the Concerns-Based Adoption Model 
provide considerable guidance to the evaluator of educational programs. When 
used in conjunction with the Stufflebeafn model, they enable the evaluator to 
describe the program completely and to relate that description to the needs of 
the appropriate decision makers. 



Results 

As noted in the introduction to this report, the primary objectives of the 
project could not be achieved, because the FACIT materials were not available. 
Therefore, a shift in the focus of the project led to the development of guide- 
lines for describing competency-based programs to be evaluated. These guide- 
lines were designed for use within any evaluation model that focuses on the 
decision-making purpose of evaluation," although Stufflebeam's rpodel served as 
a prototype of all decision-making-oriented eva-luation models. 

The following assumptions, which guided the development of the program- 
description guideTines; have implications also for the evaluatiof^ plans that 
incorporate the guidelines: 

Each competency^based vocational program differs from all others in 
the way it is implemented, regardless of any labels ttftt might be 
applied to the nature of instruction used in the program. 

Each (iompetency-based vocational program,.changes oveV^ time in\ the way 
it is implemented. . 

The outcomes and implementation of all vocational programs are influ- 
enced by factors beyond the control of those responsible for adminis- 
tering and implementing the program. . 

T h e infor m a tfo^-ebtained tH'trott^ any eval uat^tjnH^ used for maki ng 
decisions affecting the implementation or continuance of the program. 

There is np one set of techniques and strategies that is most appro- 
priate for. evaluating a competency-based vocational program. ^ 

6. The financial, personnel, and other resources used in evaluating a 
competency-based vocational program vary according to the nature uf 
the program and the purpose of the evaluation. 



Program-Description Guidelines 

The task of describing the program to be evaluated can be broken down into 
two major phases, based on the specificity of the information needs delineated 
and the use to whtch the information will be put. Description activities in 
Phase One result in a rather global picture of the program and its context. 
This global picture is most often used to plan subsequent evaluation activities. 
Ph^se Two, on- the other hand, provides rather detailed information necessary for 
making decisions:. regarding program improvement, continuation, and so on. 

1. • ' 
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Program Description: Phase One 

Most evaluation models call for a description of the program to be evaluated. 
The description serves primarily to focus the evaluator's efforts and helps to 
ensure that the client and the evaluator have a common understanding of what is 
being evaluated. The nature of the program description in Phase One will depend 
to some extent on the objectives and purpose of the evaluation. For some pur- 
poses, program description in this phase will be quite detailed, while for other 
purposes a broad general description will suffice. 

A flexible framework for describing programs in Phase One is provided by 
Stake. As Figure 2 illustrates. Stake's program-description matrix provides 
the evaluator with several broad categories ^in which to put the' information 
desired. Appendix A includes definitions and examples of these categories. 
Because the categories in. the description matrix are so broad, the evaluator is 
free to describe the. program in accordance with his or her needs and those of 
the client. Appendix* B contains a" collection of criteria and instruments that 

are related to the evaluation of competency-based vocations! prograTns. Each " • " 

set of criteria or each instrument is suitable for the Phase One description 
of one or more aspects of the program to be evaluated. 

Phase One program description is used most often in planning the evaluation. 
As such, it focuses on program intents, rather than on program observations. 
Oncfc the program is described in terms of what was intended, the evaluator can 
plan the collection of information to determine whether or not what was Intended 
actually took plac^. 

. ^ ■ ■• ^ • 

Program Description: Phase. Two . ' 

The nature of Phase Two program description varies according to the purpose 
^ of the evaluation and depends to a large extent upon what was described in Phase 

One. In terms of the broad categories of information needed to evaluate any . 
competency -based program. Stake's program-description matrix is useful. As 
mentioned above, the evaluator in Phase Two concentrates on the behavioral 
description of the program. 

In order to obtain more process-oriented information than is specified in 

. Stake '. s program-descriptiofLJnajurix.>, the methodology from the Concerns -Bssed 

Adoption Model will prove useful. Tables 4, 5, and 6 describe the main concepts 
related to information collection in the Concerns -Based Adoption Model. The 
purpose of coJlecting information about stages of concern (Table 4) or levels of 
use (Tables 5 4 6) is to determine the iiature and extent .of program implementa- 
tion and to discover possible causal relationships. 

Appendix C contains interview and questionnaire guidelines to determine the 
levels of use of FACIT. Using these guidelines in conjunction with an in-depth- 
knowledge of FACIT, the evaluator can develop the necessary instruments to deter- 
mine the nature and extent of FACIT implementation. The criteria and instru- 
ments in Appendix B can also be used to make information collection within 
the framework of the Stake matrix more precise and relevant to competency- 
based vocational programs. 
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Summarv of Guidelines 



Program evaluation cannot take place without a precise and detailed des> 
criptlon of the program to be evaluated. One approach to describing competency- 
based programs Is to break the task down Into two phases. In the first phase, 
the program Is' described with just enough detail to plan the Information- 
collection and decision-making aspects of the evaluation. Then, In the second 
phase, the evaluation plan is carried out by describing all aspects of the pro- 
gram in sufficient detail for the nature of the decisions to be made. The Stake 
program-description matrix provides an overall framework for categorizing the 
descriptive information about the program, while the Concerns-Based Adoption 
Model contains guidelines for collecting- detailed infonnation about program ' 
Implementation. Beth phases or program description are consistent with the 
generalized evaluatiK)n model of Stuff lebe&m and can be used to enhance the 
relevance of information col lection ^when evaluating competency-based programs. 



Conclusions and Recbmmendatlons 

Because of the unavailability of the FACIT materials, the focus of the 
project had to t>e altered. Thus, instead of developing' an evaluation model 
tailored to the needs of locat institutions involved in evaluating the FACIT 
materials, the project focused on the need for a generalized evalMation model 
for competency-based education. After reviewing numerous approaches to evalua- 
tion in general as well as to competency-ba^e^ education, the Stufflebeam model 
for conducting decision-oriented program evaluations was found to meet most 
criteria that are important in evaluat ing educational programs. Since; however. 
Stuff lebeam's model provides little guidance In the program-description aspects 
of evaluation', it mis decided to prepare additional program-description guide- 
lines. The guidelines that were developed represent a synthesis of the Stake 
evaluation model and the Concerns-Based Adoption Model. 

It would be premature at this point to draw any conclusions regarding the 
effectiveness or efficiency of planning evaluations based on the Stufflebeam 
model as supplemented with the program-description guidelines. However, the 
popularity of the Stufflebeam model does suggest that it will facilitate the 
evaluation of competency-based education. In addition, the Stake model and 
the Concerns-Based Adoption Model have each befen used independently as the basns 
for nu merou s evaluation studies. Thereforej^ although no firm conclusions can be 
TraWn7^conTi dehce In tTie Stuff! ebeam mb^ an^" program-description guidel ines 
would not be unjustified. • 

Several recommendations can be made regarding use of the Stufflebeam model 
and the program-description guidelines in evaluating competency-based programs. 
These recomnendations apply not only to evaluating FACIT, but also to evaluating 
programs based on any approach to competency-based education. 

• 1. Evaluators should use the program-description guidelines as intended and 
not as a sole source for ready-made evaluation instruments. That Is, they 
should develop or obtain instruments that are most appropriate to the 
program they are evaluating. Since not all competency-based programs 
operate under the same assumptions, it would be inappropriate to compare 
a program against criteria that do not reflect the intents of the program 
originators, except when such additional crfteria are consistent with the 
purposes of the evaluation. 



• 
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2. Evaluators planning to evaluate programs developed according to FACIT 
procedures should become thoroughly familiar with the conterc of the 
FACIT matertals. This is necessary because the guidelines for levels- 
of-use Interviewing do not Include behavioral statements reflecting 
FACIT implementation. These must be determined by the evaluator. 






f 

. 3. Sfnce guidelines for program description have not been empirically' 
tested, it would be appropriate for evaluators who use them to care- 
. fully document their Impact on the conduct of the evaluation. 'Although 
this should not discourage their. use, there is almost certainly room 
for improvement In the guidelines. 
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Definitions and Related Examples Useful 
In Describing Educational Programs 
for Evaluation Purposes* 



GENERAL 



Definitions 



Examples 



Antecedents : j,* 

All ^he existing conditions that are 
likely to Influence either positively 
or negatively the transactions or 
outcomes of a competency- based program 
or product. 

^Antecedents subject to change; ' Antecedents subject to change; 

Prior cdiKlltlons existing within a o The attitudes of administrators 

setting that may affect the transaction toward CBE.- 

and outcomes and that can be altered. o The placement of teachers' aides 



and other paraprofesslonals 
within the Institution. . 



Antecedents not subject to change; 



Antecedents not subject to change; 



Prior conditions existing within a 
setting that ntay affect the transaction 
and outcomes, but that cannot be altered. 



0 Socioeconomic status of the 
students. 



0 Climatic conditions needed for 
year-round Instruction In the 
building trades. 



Transactions: i. 



Activities Involved In the competency-based 
Instructional program or the use of the 
products being jevaluated-. 



Instructional transactions: 



Instructional transactions; 



those teaching/learning activities that lead 
directly to mastery of program or product 
instructional objectives; 



0 A teacher conducts a demon- 
stration of a job task. 

0 A student works through a 

module on repairing carburetors. 



Nonlnstructlonal transactions: 



Nonlnstructlonal transactions: 



Those activities designed to directly o A teacher creates a file for 

support the Implementation of a competency- each class member, 

based instructional program or use of a ' o Community volunteers are 
product. solicited for participation on 



a CBE advisory connlttee. 




btritt<ML--Loncinueo 

Definitions 



Negative transactions : 

All activities that obstruct or delay 
the implementation of competency-based 
programs. 

Outcomes ; 

All the results or consequences of. a 
competency-based program or product. 

Instructional outcomes ; 

Those cognitive, affective, or psychomotor 
skjils that have been acquired by individual 
learners as a result of the competency-based 
program or product. 



Noninstructional outcomes : 

All the results, or consequences, of a 
competency-based instructional program 
or product other than individual student 
achievement of cognitive, affective, or 
psychomotor objectives. 



'INTENTS 

Intended antecedents : 

Expectations or presuppositions about those 
prior conditions that may influence either 
positively or negatively the transactions 
and outcomes of a competency-based program 
or product. 



Examples ' 

Neqat 1 ve transactions ; 

0 A series of brmb threats are 
made toward the institution. 



Instructional outcomes ; 

0 A student masters the 'kills 
required for operating a piice 
of equipment used on the Job. 

0 A teacher in an in-service 
program demonstrates a more 
positive attitude toward CBE. 

Noninstructional outcome s; 

0 A closer working relationship 
between institutional and 
community representatives is 
attained. 

0 Student enrollment in* voca- 
tional curriculum areas 
increases by 15%. 



Intended antecedents subject to 
c hange; 

0 The initial attitudes of teachers 
toward CBE are expected to be 
positive.. 

0 Some change in the placement of 
teachers' aides and other para- 
professionals within the 
institution is planned. 

Intended antecedents not subject 
to change ! " 

0 The average family outcome of 

students will be at least 

$5000 per year. 
0 Heavy snow •and winter rains 

will be no worse this year 

than in the past. 
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INTENTS-Continued 



Definition^ 

Intended transactions ; 

All the activities planned for a 
competency-based instructional program 
'or the use of the products being evaluted. 



Intended outcomes ; 

All the goals, objectives or results 
that a competency- based instructional 
program or product is designed to 
achieve. 



Examples 

Intended instructional transactions; 

0 A teacher plans for a student 
' tiQ practice the use of a 
circular bandsaw for beveling 
fpur. planks of lumber at a 
450 angie. 

0 CBE training supervisors plan 
for teachers in an in-seryice 
program to analyze the h4erar- • 
chical arrangement df a set 
of enabling objectives. 

Intended noninstructional trans- 
actions ; 

0 A teacher plans for a proctor 
to inventory available con- 
sumable resources. 

0 CBE teachers plan for a CBE 
advisory committee to provide 
them with job-related infor- 
mation needed to solve goal- 
setting problems. 

Intended instructional outcomes; 

0 Given the needed. student-related 
and job-related information, a 
teacher in an in-service train- 
ing program will !■« able to 
solve student go^. -setting 
problems. 

0 Given a group activity that 
requires cooperation among 
group members, t'le student 
'Will choose to contribute to 
the efforts of co-workers by 
answering their questions and. 
assisting in their assigned 
tasks. 

Intended noninstructional outcomes ; 

0 The number of students who fail 
to graduate from a vocational 
program will be reduced by 50% 
over the next two years. 

0 Teachers will share consumable 
and nonconsumable resources 
among themselves for the pur- 
poses of economy and efficiency 
in instruction. 
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OBSERVATIONS 



Definitions 
Observed antecedents ; 

Actual prior conditions as indicated by 
relevant information or records, regardless 
of whether these conditions have been 
expected or assumed. 



Examples 

Observed antecedents subject to 
change : 

0 90% of teaichers have ^ 
positive initial attitude 
toward C6E, as indicated by 
the results of questionnaires 
and interviews. 

0 Seven teacher's aides and other 
paraprofessionals are available 
to help in meeting the require- 
ments of managing CB.E classroofRS, 
according to Ue records of 
the insliitution. 



Observed transactions: 

All planned or unplanned activities 
that actually take p'lace in the program 
or all uses of the products- being 
evaluated. 



Observed antecedent not subject 
to changeT 

0 The average family income was 

found to be $3,500. 
0 Snow arid heavy rains prevented 

work on outside construction 

projects on 60% of the school 

days during the term. 

Observed instructional transactions ; 

0 A student bevels three planks 
of 2" by 4" pinewood at a 47° 
angle using a circular bandsaw 
during a forty-five minute 
period while a trained tutor 
monitors the student and pro- 
vides feedback. 

0 A group of four teachers . 
arrange an -unorganized set of 
enabling Objectives within a 
> procedural hierarchy during a 
sixty-minute in-service group 
activity period, which is 
monitored by a CBE training 
supervisor. 

Observed noninstructional trans - 
actions : 

0 A teacher's follow-up evaluation 
fi-le indicates that (s)he con- 
ducted a 30-minute interview 
with the employer of a former 
stddent to gather information.- 



OBSERVATIONS-Contlnued 



Definitions 



Observed outcomes: 

All. the actual positive and negative 
consequences of a <:ompetency- based 
Instructional program or product. 



Examples 

A report submitted by a 
adViiory committee to ta^ 
who wiitK to help studenli 
goal-set^ng problems d( 
the availability of Jol 
In Industrle^that supi 
the surroundlHo commun1iy> 



Observed negative t> 



Ions: 



0 According to a poll of the 
student body, 25% of the 
Institution's students are 
under the Influence of an 
Illegal drug during the 
majority of school days. 

0 Durln^,,*^ teachers' strike, 
653; of Ijfie institution's 
facuUy did not ^report to 
work, for eight s<5hool (<ays. 

0 A recession In the national 
economy causes aVtut In 
funding to school\ systems. 

Observed Instructlohal outcomes; 

0 In, three out of five group 
activities that a.re monitored 
by a trained tutor, k student 
chooses to answer thf questions 
of his co-workers and to assist 
In their assigned tasks. 

0 During an In-service group <- 
activity that Is monitored by 
a CBE training supervisor* a 
teacher resolves a f^pothetlcal 
student ' s goal -setting prObl ems 
to the satisfaction of his group 
members four out of five tIAies. 



Observed nonlnstructlonal outcomes: 

0 The student drop-oAJt rate In an^ 
Institution's vocational curric- 
ulum decreases by 20% during 
the first two years of CBE « 
Implementation. 

0 Two teachers resigned their 
positions In opposition to CBE 
at the end of the first year 
of Implementation. 
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STANDARDS 



Definitions ' 

Stai |ards for, antecedent s; 

Crit^U-*h«ri5iNi(ify different degrees 
of c^gruence betw<9en intended and 
obser^d anteceder^s. 



-Standards for transactions; 



Criteria that specify the allowable 
differences between intended and 
observed transactions. 



Standards for outcomes: 

Criteria that specify different degrees 
of congruence between intended and 
observed outcomes . 



-Examples 

Standards for antecedents subject 
to change ; 

0 85% of an institution's 

teacjiers must display a positive 
attitude toward QBE. 

0 The number of necessary teachers' 
aides and Other paraprofesslonals 
within the particular institution 
should be no 'less than 13. 

Standards 'for antecedents not 
subject to change; ~ 

0 The average family income 
shou^ld be at least $4,500. 

0 Participation in tne pilot- 
test project requires that a 
program use the new materials 
on at least 80% of the school 
days. 

Standards for instructional 
transactions :" ~ 

0 The student must bevel the 
wood within 1% of the assigned 
angel without marring it, and 
the assignment must be com- 
pleted within 30 minutes. 

^ Standards for noninstructional 
transactions: 

0 The advisory committee must 
make available to teachers the 
required job-related infor- , 
mation within their areas of 
'expertise. 

Standards for negative transactions; 

0 The teacher strike must not 
disrupt the training of 
students. 

Standards for instructional 
outcomes: 

0 A teacher's resolution of a 
hypothetical student's goal- 
setting problems during an 
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STANDARDS-Contlnued 



Definitions 



Ex amples 

Standards for j ns true t i ona 1 
ou'tcofneV : contTnued 

in-service group activity 
must be done to the satisfaction 
of group members with 80% 
consistency. s 

Standards for noninstructtonal 
outcomes ; 

« 

0 The introduction of CBE in a 
, vocational curriculum will 
decrease the student drop-out 
(•ate by 50% during the first 
two years of implementation. 



JUDGMENTS 



Judgments regarding antecedents ; 

The decisions that specify any needed 
courses of action on the basis of what 
is known about antecedent conditions. 



Judgments regarding antecedents 
subject to cnange : 

0, The initial attitudes ot 
teacher?* toward CBE are posi- 
tive, and no action to alter 
initial attitudes need be 
undertaken. 

0 The number of available 
■teachers' aides within the 
institution is six short of 
the number needed to help 
meet the requirements of CBE 
classroom management; action 
should be ta^en'to hire six 
more teachers' aides. 

Judgments regarding antecedents 
not subject. to change: 

0 Since the average family 
income was below the standard, 
it is recommended that students 
in the special programr not be 
required to purchase the course 
materials. 

0 The building trades program 
should be dropped from the 
program, until next year. 



JUDGMENTS-Contlnued, 



OefinHlons 



Examples 



Judgments regarding transact ioivs ; 

The decisions that specify any needed 
courses of action on the basis of what 
is known about the quality of completed 
transactions. 



Judgments regarding outcomes; 

The decisions that specify any needed 
courses of action on the basis of what 
is known about current achievement of 
Intended outcomes. 



^ Judgments regardlng^ Instructional i 
transactions; 

0 The student did not complete/ 
the activity Qn beveling wood 
' as planned; action should be 
taken to evaluate and possibly' 
revise the Instruction pro- 
vided for this objective, and 
to provide the student with 
additional Instruction^ 

Judgments regarding non Instructional 
transactions; ^ 

0 The CBE advisory committee did 
not complete the activity as 
planned. because job require- 
ment's were not reported to 
teacners; action should be 
taken to Inform the committee 
about Its functions, and to 
obtain job-requirements 
Information. 

Judgments regarding negative 
transactions ; \^ 

0 The teacher ktrike disrupted 
the training! of sMents; action 
should be ta)('en to avoid strikes 
In the future and to provide 
the .Instruction that wa^s delayed. 

Judgments regat^dlng Instructional 
outcomes; 

0 The objective that deals with 
a teacher's ability to solve 
goal -sett Incf problems was met; 
no action to evaluate and revise 
the Instruction for this 
objective need be undertaken, 
and no additional Instruction 
for this objective need be 
provided to this teacher. 



JUDGMENTS-Continued 



Definitions 



Examples 



Judgments regarding nonlnstructlonal 
, outcomes ; 

0 The criteria for achieving 
the Intended outcome of 
reducing student drop-out 
rate have not been met; 
action should be taken to 
evaluate and revise the 
Instruction aimed at this 
outcome. ^ 
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APPENDIX B 

CrUerIa and instruments for Evaluating' 
Competency-Bastfd Programs 
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Competency-Based Program Assessment Instrument 

(The Instrument that follows Is designed to be used by program designers 
and fmplementers 1p evaluating the degree to which competency-based education 
has been Implemented. Five areas are addre&sed: competency specifications, 
instruction, assessment, governance. and management, and overall program 
effectiveness.) 



Source: Burke, J. B., et al. Criteria for describing and assessing competency 
jased programs . New York: Multi-State Consortium on Performance-tElased Teach^ 
Education, no date. 7j 



AMPLIFIED FORMAT 

4 

Competency Specificationt 

t.O Competencies are bated on an analysis of the professional role(s) and/or a theoretical 
1 formulation of professional responsibilities. 

Indicators: 1.1 Rationale for program model and competencies is written trir; 



concrete and definitive ^ ~ no* written 

1 .2 Assumptions about l^feimef*rttofessional role, program constraints, 
and learning and instructidrfvprinciples explicated 



yes "O 

1.3 Each competency in program can be logically linked to program 
model 



all 100% 50% • no 

competencies comp. comp. comp. 

1.4 Program personnel who designed program can describe rational 
and link competencies to model. 



all personnel only small core of developers 

1 .5 E ntire program conceptualized as an integrated whole 



total program specific partt specific parts 

conceptualized designed; ovflrlap d<>«igned 
then specific ' and gaps formed 

parts developed by analysis, then 

linked together 



1.6^ 



1 Each criterion has an Indicator added to facilitate the study of indicators unique to specific 
programs. Users are encouraged to ad^as mdny as are appropriate to the situation. 
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Competencies are specified and made public prior to instruction. 

Indicators: 5.1 Required competencies and options are known to learners as 

they enter progran^^ 



Written statement Written requirements Students cdn not 

of competencies are available to student describe the program, 

and diagnostic prior to each program its competencies, 

procedures are part. and their options, 
provided student as 

he enters program. « 
5.2 Ail required competencies are specified prior to initial instruction. 



All specified most competencies written 

and published as program implemented 

5.3 Indicators of competence vary among individuals and from 
setting to setting. 



Flexible indicators Rigid indicators 

as appropriate required of ill 

5.4 



Learners completing the CBE program demonstrate a wide range of competency 
profiles. 

Indicators: 6.1 Both required and optional competencies are included in the 

program. 



many options open to students no options 




6.2 Individual learner needs dictate program emphases. 



6.7 



always generally never 

6.3 Required competencies and options are made known to students 
in ac^vance. 



always generally ^ never 

6.4 Program options are not closed. 

y«s no 

6.5 Students may choose program options. 



yes no 

6.6 Learner cognitive styles, teaching setting, area of instruction aid 
in determining program options. 



yes no 



Instruction ' 

7.0 The instructional program is derived from and linked to specified competencies. 

Indicators: 7.1 Competencies determine the learning outcomes to be acquired. 



All learning outcomes Activities are not 

(knowledge, skills, etc.) related to specified 

of the instructional program competencies, 
are derived and linked to 
specified competencies. 
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7.2 



* Activities provided for tht student to use In ecquiring the competencies 
ere determined by the nature of the competency: (i.e.. One does 
not learn problem solving Skills from expository teKhingK 



Activities are derived from Theielitionihlpi 
and linked to the com- between activities 

petencies to be acquired. and the competency 

being acquired are 
not evident. 

7.3 The elements in evaluation instruments are directly related to 
specified competencies. 



Each element of There appear to be no 

studen: evaluation relationships between 

Instruments is directly the items in student 

tracable to a specified evaluation instruments 

competency ahdxSpeclfled 

competencies 



7.4 



Instruction which supports competency development is organized into units of manageable 
size. 

Indicators: 8.1 The size of the instructional unit is dejiendent upon program variables. 



The size of the The size of the instruc- 

instructional unit is tional unit is not 

related logically to loflically established, 

appropriate program The size varies widely. 

variaMet. 

8.2 Instructional units are organiited and partitioned to provide data 
and feedback on learner's stage of development. 



At the end of each instructional The unit size is not 

unit the learner is given feed- related to the student's 

back on progress. feedback needs. 



8 3 LMrntr'i •xp.rltnce with Instructional uniti. it ui«d to 
dtttrmina suitability of unit sizt. 



Student's fetaback 
concarning tht suit- 
ability of uniu (by 
langth, complexity, 
amount of content, 
etc.) is used to re- 
vise units. 



No attempt it 
made to obtain 
knowledge 
of student's ex- 
perience in using 
units. 



8.4 



Instruction is organlied and implemented so a^o accommodate learner style, 
sequence preference, pacing and perceived needs. ■ ^ 

Indicators: 9.1 Instruction provides alternative learning activities. 



vldesuggest^ aHernate for individual students' 

riticro:m:5.t.th. -n*^- 

studentt' learning style. 
9.2 Program sequence includes a wide range of options. 



FrogTam sequence options are 

known by learner. known or available 

to learner. 

• * * 

9.3 Initnietion It pK»d to th« Itarntr. 



LMrn^^iprocMd Som. dmtr.nti«iof. !■.«/.."*« 

.tvarviiUPKH ii mid. In Uirntf procMditthi 

oJth.pfOjr.m. «lprlm»ilyby through th. 

learner circum- program. 

stances rather 

than program design. 
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9.4 Inttruction providM for learner perceived needi. 



Inttructionalunits 

Instfuc 0 ^ p 

include "l^er for achieving competenc* 

lelect* ' options for "ot available to 

Inttruction. program. 

9,5 The learner it given opportunitiet to attett ef f ectlvenett of hit preferred 
learning ttylet. 



there are opportunitiet Little or no attention 

for the ttudent to It given to the relative 

closely examine with ' effectiveness of particular 

technical and pro- 'wnino ityles as they 
fessional assistance applied by particular 

. the learning styles individuals in reaching 

preferred. their objectives. 

9.6 Conferences are held with learners at prescribed intervals. 



Know schedule of No conferences 

conferences combined 

with open system where :A. 
conference really held 
when iieeded. 
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10.0 Learner progress is determined by demonstrated competence. 



10.1 The student is knowledgable of the general nature of competencies 
and criteria used to determine the extent to which performance 
approaches professional standards for acceptability. 



Student describes 
competencies apd 
the standards for 
acceptability. 



Student not »b\% 
to describe com- 
petencies request 
or criteria that 
are acceptable. 



ERIC 



42 



10.2 



Learner progress records ere edequately deteiled in terms of 
the competencies to be acquired. 



^.earner progress records Learner progress 

are adequately deteiled« records not kept 

in terms of the com- on file, 

petencies to be acquired. « . . 

10.3 > Learner progress records are used to chart future programs' 
directions. 



Learner progress records Learner progreu 

are. frequently used to records ere seldom 

chart program direction. if ever used to chart 

program direction. 

10.4 The demonstration of progress in acquiring the competency is 
the focus of attention in determining the extent to which the 
learner is experiencing success. 



Success is determined by Success is determined 

f extent of progress by some other other 

in acquiring the com* ^-^ criterion such as amount 

petency. of knowledge ecquired. 

or number of ectivities 
completed. 

10.5 The instruction management system makes provisions for students 
to be working at various points of development concurrently. 



Instruction is modularized 
and organiziid to be carried 
out individually or in small 
, groups by variable scheduling 
: techniques. 



10.6 



Instruction is besed 
on the assumption 
that all students 
should acquire the same 
learnings at the same 
time. 
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1 1 .0 The extent of learner's progress in demonstrating competencies is made known to 
him. thrc^ughbut the program. 

Indicators: 11.1 Learner progress records are maintained and available to all 

concerned (learner, instructors, counselors). 



Learner progress records 
are accessible, adequately 
detailed, and open to 
himself. insWctors, and 
counselors. 



Learner progress 
records are in- 
accessible, in- 
adequate, and/or 
closed to students. 



1 1 .2 The instructional staff (instructors and counselors) and learner 
periodically review progress records in conference. 



Student progress 
conferences are 
held frequently. 



Student progress 
conferences are 
non-existent. 



1 1 .3 The instructional management system provides for the frequent 
andA>r continuous updating of the student's pr^ogrHS records. 



Progress records updated 
on a continuing basis. 



Progress records if 
available are only 
updated at infrequent 
(i.e.. semester end) 
periods. 



1 1 .4 The student is provided with opportunities to acquire skill in 
analyzing^ and evaluating his own pi'ofessional behavior. 



In addition to being 
provided with information 
about his progrHS. the 
student is helped to 
acquire skill in analyzing 
his own professional 
behavior. 



Little or no at ' 
tention is given 
to the analysis of 
the student's progress^ 
and none in helping 
the student acquire 
this skill himself. 



11.5 
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Initructional iptcificationi art raviawad and raviiad batad on faadback data. 

Indicatori: 12. 1 Spacificaf ioni for tha initructional lyitam ara axplicit and 
all conearnad (itudantt, Jhitructon, countalori. initructional 
profaiiional larvicat parionnal. ate.) ara awara of thaia 
ipacifipationi. 



A lilt of ipacificationi 
for tba initructional 
lyitam ii publiihad. 



Naithar ipacifica* 
tioni nor policial 
concarning tha 
initructional lyitam 
hava baan ratordad < 
much laii mada- 
known tothoia ' 
involvad. 



12.2 Procadu rai Hava baan aitabliihad for having itudanti aiiau 
tha initructional lyitam. 



On a fraquant pariodic 
or continuing baiii 
Itudanti ara aikad to 
raKt to tha affactiva- 
nau of tha procadurai 
uiad in tha initrue* 
tidnal lyitam. 



No attampt ii mada 
to obtain itudanti' 
raactioni to tha in- 
itructional pro* 
cadurai. 



A Ni^ida ranga of data it coniidarad in tha analyiii of tha in- 
itructional lyitam. (Studant tima, initructor tima. in- 
itructional raiourcai, ntanagamant naadi. laarnar parformanca, 
ate). 



An axtaniiva collaction fy|o attempt ii mada 

of data ii uiad for tha toanalyaatha 

analyiii of tha initruc- operation of tha 

tional lyitam. initructional lyitam. 

» 

12.4 Data obtained from tha analyiii of tha initructional lyitem ai 
provided by itudent feedback ara uiad to reviie the lyitem. 



On a frequent periodic No lyitematic or regular- 

or continuing baiii tha attampt ii made to reviia 

initructional ayitam it the initructional lyitam. 

reviled from data pro- Changei ara made primarily 

vidad by itudent feadbMk. on demand from tome con- 

4 ' dition or authority. 
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12.5 



Aitettment 

13.0 Competency measures are related validly to competency statements. 

Indicators: f^.1 A listing of performance indicators is included with each com* 
patency statement. 



Multiple indicators Few competency No competency 

are present for all statements have statements . 

competency state* multiple have more than 

ments indicators one Indicator 

13.2 Indicators are logically related to competency statements. . 



all some none 

competencies -of them 

13.3 Measuring Instruments are logically related to indicators. 



all some none 

competencies of them 



13.4 



14.0 Competency measures are specific, realistic, and sensitive to nuance. 

indicators: 14.1 Competency measures discriminate between learners who 

demonstrate and those who do not demonstrate competency. 



All ri^easures Most measures Undetermined 

moft of the time most of the time 

14.2 Measures assess consistency of performance over time. 
Always Usually Seldom 



14.3 Reliability of instrumenti It known and high. 



Computad for 
all Inttrumanti 
and high 



Soma 

inttrumanti 



Not known 



14.4 Procadurat for maaturihg compttancy damonttratlon ara 
tpacifiad to at to attuma quality and contittancy. 



14.6 



Qanarally followed 
and known by data 
cbllecton 



Procedure! not 
tpecified, known 
or.followfld. 



14.5 Data collection proceduret require reallttic time and retourca 
expenditure! by ttudentt and ttaff. 



Realittic 



Unreali!tic 



\ •• . ■ ^ 

0 ^ppmpetency meature! diicriminata on the batit of ttandardt tet for competency 
d^onttration. 

' ln(|i6atort} 15.1 Specific acceptable ttandardt are ettabllthed prior to competency 
^ demonttration for all competenciet. 



Standardt are tet 
and made public. 



Some ttandardt 
are tet In advance 



15.2 Standardt are bated upon data. 



Standard! ara depen* ' 
dent upon Individual 
catf , decided after 
competency demonttra* 
tion. 



Logic, data or 
retearch it uted 
at batIt fdr 
ttandardt. 



Standardt ara 
pretent but 
primarily 
bated upon 
judgment or on 
negotiation among 
developers. 



Standardt are un« 
known or dependent 
upon Individual 
catet. 
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15.3 Competency mevures provide data\indicatrng tbe extent to ^ 
which standards are met. 



For all For some For no 

standards standards standards 

15.4 Standards are realistic expectations of professional developmental. 



AM standards Some standards Standards are not 

appropriate for are appropriate . realistic for particular 
particular phase 2 • phas^ 4 program to 

professional yyhiqh they are applied, 

development. 



15.5 Standards are applied based on the demonstration context. 



Stahdiardi may be Standards are* Standards are nfln* 
negotiate prior. modified in in* resistant or rigidly 

to demonstration. dividual cases after applied 

competency demon* 

jtration attempted. 



^ 15.6 



ata provided by competency measuret^are manageable and useful in decision making, 
dicatort: 167T^ Data are collected and stored in an easily retrievable form. 



Data on competency Some data are Not col^pcted or 

measures are col- collected, storai^ not stored, 

lected and centrally not planned or 

stored. centrally located. 

16.2 Data are reported at pre specified decision points. 



Reports are helpful Some reports are No reports are 
to decision makers made as a result generated. 

of special needs. 
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16.3 Data are used in making programmatic decisions 

Ata are generated Occa^onally data Not used, 
as a basis for de- used al a basis 

cision making. for decision 

making. 

16.4 Data collection and analysis procedures are feasible in terms 
o.f time, personnel, and resources. 



Efficiently handled Collected but Burden is on 
within resources. seldom used program, or 

because pro- ] not collected, 
ceduresare 
ciMffbersome. 



/ 

y . 16.5 Data are easy to interpret 



Format of data Not easily 

analysis is clear. interpretable. 



16.6 . 



17.0 Competency measures and standards are specified and mjde public prior to instruction. 
Indicators; 17.1 Competency measures and standards are in a written form. 



For all competency Some are Npneare 
rneasuretand available. written, 

standards. 

17.2 Competency measures and standards are specified in advance. 



yes for some no 

17.3 Students can describe competency measures and standards. 

• ' - •--/ - ■ ^ 

all known 10 them somejknown unknown to 



\ 1 student 



17.4 Procedures for demonsUating competencies are known to , 
students and faculty. . 

'V. * • - 

' Known to all. . . ^^nown to some. Unkn|6wn to 

' students. / 



17.5 , 



Governance and Management 



■18.0 Policy s|ptements are written to govern, in broad outline, the intended structure, 
content, operation and resource base of the program. 



Indicators: 18.i1 



A formfilly recogniied poiicy*|naking or governing body exists 
for the program. 



A governing body is 
recognrted as having 
responsibility and 
^uthb^ity for making 
policies^ for the 
pr»gra<ri: ^ 



No authority recognized 
to which one may turn to 
obtain knowledge of 
existing policies upon 
which to base program 
operations. 



18.2 All instjlutions^ agencies, organizations, and groups participating 
progrj 



in the program are represented in policy decisior^s that affect 
the program. 



18.3 



« When^ policies are forhied- 
^a)J oersons or groups which 
' ni&ybe affected by those\ 
^ (Policies ar^Mresented. 



No policies or policies 
made by one group. 



lions are supported by and made after consideration 
program effectiveness and resources required. , 



Data are collected, and 
systefnatically stored, 
and considered in re* 
'viewing, changing or 
creating policies. 



No research base exists 
for policy decisions* 
Policies are the result of 
power relationships 
and personal opinions. 
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18.4 



An explicit itatement of policies for management and governance 
of the program is available to all involved or concerned. 



18.5 



18.6 



Such a statement of policies 
is in printed form, current 
and frequently referred to 
by persons involved in manage- 
ment or governance of the 
program. 



There appears to be no orderly 
statements of policies avail 
able to persons involved in 
management or governance -of 
the program. 



Associated with the statement of policies for management and 
goverr\ance of the program is a list of the competencies 
specif ted to be demonstrated for exit from the program. 



Thefe exists a manual or ^ 
handbook which presents 
statements of competencies 
specified to be demonstrated 
for exit accompanies' by 
interpretative narr u . ). 



The expected outcomes of the 
program are not clearly 
available even in general 
statements. 



Policies, organization, and management procedures are readilv 
modified and regularly teviewed. 



Process known to all; 
review process regular. 



No known governance 
structure or a rigid, 
unmodifiable one. 



18.7 



19.0 Management functions, responsibilities, procedures and mechanisms are clearly 
defined and made explicit. 

Indicators: 19.1 Management decisions reflect stated program philosophy 
and policy. 



When management decisions are . 
made, the decision is accompanied 
by a rationale which cites the 
program policies and/or assump- 
tions upon which the decisions 
are made. 



When decisions are made 
they are primarily forced 
by urgent conditions and 
represent an arbitrary solution 
derived from political rather 
than rational interaction. 



ERIC 
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19.2 The person or group with responsibility for decision making 
has the authority and resources to implement the decision. 



No person or group is required 
to implement a management 
decision unless provided with 
the authority and resources , 
needed to ulfill th^ requirements 
of the decision. 



Frequently persons or 
groups are asked to imple- 
^ ment plans for which they 
^ have neither the resources 
nor the authority. 



19.3 Program management and governance operations are designed 
to model the characteristics desired of schools and claL^room 
in which program graduates will teach. 



The criteria established for the 
management and governance of the 
teacher education program repre- 
sents the kind of management and 
governance program which would 
be desirable for the schools in 
which the graduates are likely 
to teach. 



The criteria used for 
assessing the manage- 
ment and governance of 
the teacher education program 
differ from those thought 
suitable for the schools 
\p which the graduates 
are likely to teach. 



19.4 Job definitions, staff selection, and job assignment respon* 
sibilities carried out by the same management-governance 
teams who are entrusted with other management-governance 
functions. 



The preparation of job des- 
criptiobs. the selection 
of staff and the assignment 
of persdnhel to tasks is 
a function of the manage- 
ment-governance team. 



Various individuals in 
management tre entrusted 
with various management 
functions. There is little 
or no attempt to co- 
ordinate. 



19.5 Formally' recognized procedures and mechanisms exist for 

arriving al the various levels of program management decisions. 



Procedures for program manage- 
ment decisions and made public, 
used consistently, and 
acceptable to all involved. 



Procedures for program manage- 
ment decisions are in- 
consistently followed. 



erJc ^ a 



19.6 



Total Program 

20.0 Program staff attempt to model theattltude* and behaviors desired of students In 
the program. 

Indicators'. 20.1 Faculty and staff meet regularly to work as teams. 



Always ^ Sometimes Never 

20.2 Staff treats students with the respect and concern for support 
whicli is of the same high quality expected of graduates with 
their relation to school pupils. 



Always - Sometimes Never 

20.3 Staff members openly share differences of philosophy and 
social positions so that students see the appropriateneu 
and strength in diversity. 



Always Sometimes Nevei 

20.4 Instructional staff use the CBE principles in their own teaching. 



Yes 



No 

20.5 \ 



\ 



21.0 Provisions are made for staff orientation, assessment, improvement, and reward. 
Indicators: 21.1 Personnel training programs are competency-based. 



Improvement of Isolated No organized 

program personnel activities. training program, 

is through a CBE 
designed system. 
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21.2 Evaluation profiles are kept on all staff and made available 
to them. 

^W- Yei ■ • ~ " '"'"~No " 

*< 21.3 Faculty reward structure consistent with CBE role descriptions, 

requirements and developement. 



Yes . No 

21.4 Staff development activities are recognized as important at 
teaching, research; and publication. 



Yet No 



21.5 



22.Q Research and dissemination activities are an integral part of the total instructional 
system. 

Indicators: 22.1 A research strategy for validating and revising the program it 

operational. 



Written proceduret, Some efforts to Not bein^j 
hypotheses, date; study results of done, 
systematically applied, program. 

22.2 Reports of Completed studies are used in revising program. 



Numerous written Data or unwritten No reports. 

reports available, reports available. 

used. 

22.3 Research management system is operational. 



Yes, comprehensive. Some processes. Not 
workable, working. not systematic operational. 
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22.4 Procedure for iharlng reiulti with other programi and for 
obtaining their rtporti are oparational. 



Regularly iharel with Haphaiard iharlng No relatlonihip 
at least two programs, of results with other than casual 

some sharing with other programi ones, 
ten others. 

22.6 Staff can describe the research strategy, pn going studies, and 
conclusions of previous efforts. 



All staff Some Only for studies 

he Is engaged In. 



22.6 



23.0 Institutional flexibility Is sufficient for all aspectt of the program. 

Indicators: 23.1 Resource allocation Is based on student outcomes rather than 
course competencies. 



Resources allocation Resources allocated 

determined by objectivet . by course enrollments, 

completed by students. 

23.2 Additional resources (personnel, materials, facilities, funds) 
are provided for program development. 



30% or more 15% Increase None 

Increase for In resource 

program design. (personnel^nd 

dollars). \ e,^ 

23.3 Resources are contributed by all consortium members (school 
districts, colleges, professions) to collaborative effort beyond 
Individual Institutional needs. 



411 partners contribute At least one Institution No additional 
finds and personnel to provides additional funds provided. 
\\d consortium. funds. 



ERIC 55 9 



23.4 Course, grading, and program revision procedures support the 
tentativeneis necessary to compliment the program. 



Changes readily 
accepted on 
experimental 
basis. s 



Involved proce- 
dures and numerous 
authorizations by ' 
committees on ad* 
ministrators 
necessary for 
changes. 



No changes 
possible. 



23.5 



24.0 The program it planned and operated as a totally unified, integrated system. 
Indicators: 24.1 The program was planned as a totally integrated system. 



\ 



Total program 
designed prior 
to independent 
parts. 



Courses compiled Independent 
into a program. parts grouped 
together and 
called a program. 



24.2 The program is operated as a system. 



Decisions reflect 
consideration of 
the total systerp. 



24.3 Management is by objectives. 



Many 
isolated ■ 
independent 
decisions. 



Yes 



Somewhat 



No 



24.4 Evaluation system provides continual feedback to assess 
objectives achievement for various sub systems. 



ERIC 



Data available 
and used, Program 
revised. 



Data occasionally 
used. 
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None 

operational. 



24.5 



When making deciiioni on one phase of the program, imflact 
on other lub-iyitemi it calculated and considered. 



Always Sometimes Never 

24.6 The sub-systems are continually being modified. 



24.9 



Yes Somewhp.t No* 

24.7 Harmony in principles among various sub-systems is apparent. 



Internal Consistency can No consistency, 

consistency be generally or not 

easily apparent. identified. considered. 

24.8 The program it continually evaluated against the actual pro- 
fessional needs, and refined based on feedback. 



Formal review - Program not 

structure amenable to 

operatic al; modification, 
changes continually 
being considered. 
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Do you Really Have a Competency- Based Program? 

4 

A Personal Self -Check 



(The Instrument below presents some general characteristics and products 
that should be part of a competency-based vocational program. Included are 
Items that refer to program antecedents and transactions. This Instrument may 
be used to describe Intents and, as a checklist for observations. Discrepancies- 
"no" responses— may indicate where further development or revisions are needed.) 

Adapted frm Hir^t, B. A., Jr. The components of competency-based vocational 
education. American Vocational Journal , November 1977, 52, 32-3^. 

Yes No 



1. Do I have information on employment opportunities? 

2. Is this information current (less than two years old)? . D L-J 

3. Do I have information on future employment opportunities for rn Q 
my students? 

4. Does my student placement record support the need for q □ 
continued training? 

5. Do I have specific job titles identified for my program? □ Q 

6. Have the specific tasks and competencies for these job r-i n 
titles been identified? -» 

7. Do I have specific lists of tools, equipment and supplies p] Q 
needed to prepare stt( ^nts for these job titles? 

8. Do I know whether the t«ks I teach my students are still □ □ 
performed on the job? V ^ 

9. Do I know how each task ranks in terms of the amount of □ □ 
worker time each consumes? 

10. Do I know how each task ranks in terms of its level of H! □ 
difficulty? 

\ 

11. Do I know which tasks are performed by workers of every \ D [2 
level during the first 12-24 months on the job? 

|12. Do I consider these factors (Questions 8-11) as I establish 
the relative Importance of the tasks I teach my students to 
perform? « 



13. Do I have a specific performance objective and a job-based p [n 

measure for success for each task my students are to \ 
perform? \ 



Yes No 



1 

14. 


Do the materials I use "talk about," "show examples," and 
Drovlde "Vearner oractlce?" 


□ ■ 


□ 


15. 


Do I collect Information (on student- performance) that 
indicates the prob'Me cause of failure of any materials/ 
media to bring stuuents to the job-based measure of success? 


□ 


□ 


11. 


Do I develop new materials and media if my students have 
difficulty meeting the Job-based measure of success? 


□ 


n 


17. 


Do I have a means of keeping my task analysis up to date 
with changes in the jobs for which my students are preparing? 


□ 


n ■ 


Other questions you may Want to jot down fot your own self- evaluation and 


use* 


18. 




□ 


□ 


19. 




□ , 


□ 






20. 






□ 






(All answers should be ^ if you want a good competency-based program.) 

1 ^ 
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Goal Clarification 

(The following presents some Items that may help the evaluator determine 
the actual goals of a program or product to be evaluated. On occasion, the 
goals "Stated or Implied for a program or product differ from those of the 
Institution where they are to be Implemented and from those of the Individual 
users (teachers and students). An evaluator should be. aware of this discrepancy, 
as It may have a severe Impact on transactions and outcomes.) 

Source: CDS Project 

1. What are the expressed goals for the program or product? 



2. What are the implied (not stated) goals, as apparent from an analysis of 
the materials? 



3. What .are the discrepancies, if any, between stated and implicit goals for 
the program or product? 



REVISED GOAL 



4. What are the stated goals. of the institution in terms of the program or 
product? , 
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5. What are the specific objectives oif the Institution In tenns of the program 
or product? ' : 



6. Are there discrepancies between the Institutional goals and objectives and 
the revised goals for the program or product? , 



INSTITUTIONAL GOAL 



7. What are the stated goals and objectives of the classroom In terms- of the 
program or product? \ 



8. How do classroom goals and objectives differ from the goals of the Institution 
and the program or product? 



CLASSROOM GOALS 



9. If goals differ, what are the likely effects on- 

t 

INTENTS? 
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9. If goals differ, what a^e the likely effects on- (continued) 
TRANSACTIONS? 



OUTCOMES? 



ERIC 
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♦ Program-Evaluation ChecfcVlst 

• . 

(The following program-evaUiatlon checklist contains variables that should be 
Included In the evaluatlQn of Hcompetency-b^sed program. The variables listed 
here are general and relate only to the program Itself. The checkllstjdoes not 
provide guidelines for specific data to be collected, nor does' It addr^ess In 
detail Institutional oij^ person variables.)" ^ . ' • 



Adapted from Andreyka, R., & Blank, B. A checkl Ist^for.t^e evaluation of competency 
based teacher^ducatl on programs. Educational Technology , January 1976, 16^, 36. 



A 
N 
T 
E 
C 
E 
D 
E 
N 
T 
S 



T 
R 
A 
N 

S 
A 
C 
T 
I 
0 
N 
S 



0 
U 
T 



\l. Program design and development 

A. Has the need for the program been Identified? 
; B. Was the design- field based? ; 

C. Are certification requirements met? . * • 
* s D. Are exit requirements eriiphaslzed? 
II. Competency Identification 

A. Is the program based^n competencies? 

B. Have the competencies been validated? 

C. Are competencies stated? ' 

III. Competency-assessment criteria y 

A. Were criteria derived from, competencies? 

B. Are level and conditions stated? 

C. Have criteria been stated' In such a way that 
they can be assessed objectively? ' 



Yes 



IV. Learning activities. 

A. Are they focused on competency attainment? 

B. Ace they field based? 

C. Are they available In different modes? 
D'. Are they individualized? 

V. Cpmpetiency assessment procedures 

A. Are pre-assessment procedures included? 

B. Is assessment based primarily on performance? 

C. Is assessment carried out in a realistic setting? 
0. Are the as:sessors qualified? , ' 

VI. Program implementation and operation- 

A. Is the totaiV program competency based? 

"87 Does the program have faculty/administrattve 

support? ■ 
C. Is the program field based? 
0. Is progress determined by mastery rather 

than time? 



VII 



No 



Don't 
Know 



I 
1 



Program evaluation • ^ 

A. Is follow-up planned for? ^ 

B. Is field-based evaluation included in the 
program? , 

C. Is evaluation used for prograTfi improvement? 



■An Instrument for Collecting Expert Judgments about an Educational Program 

(The following phresents questions to be Jsked about anteceden'ts, transactions, 
and outcomes of a program as It Is being ^ut into o'pe.ratlon. ) 



Adapted 'fviom Leiden, J. Expert Judgments as evajuatlon data. In A. lewy (Ed.), 
Handbook of curriculum »<^a1udt1on . Paris, France: UNESCCf, 1977. (See pages 173- 

' 



• 




y 




* .. 1 


Don ' t 








Yes 


No ' 


Know 


*: — : — 


A. OBJECTIVES 








* 


• 1'. 


Does the-^system Include a statement of 










1 


Instructional objectives? 










2. 


Is this statement detailed enough to be 








A 




helpful? s : • - 








N 


• 3 • 


Is this statement directed to both teacher 








T 




and students? 








' E 


4. 


If the system's objectives are unstated, are 








C 




' they obvious? 








E 


'5. 


Are the objectives readily compatible with 








0 




Individualized or self-paced Instruction? 








E 


6. 


Do the objectives "Invite the student's 








N 




affective Involvement? 








T 


7. 


Do the objectives appear to Involve only 




• 




S 




cognitive development?, '■ 










8. 


Are the objectives conceived In a tight 




• 








relation to the subdect matter? 










9. 


Are the objectives conceived in relation 












to a student's human development? 






> 


r 


B. MANNER OF USE. : / 




4 


1 

t 






Are all elements of the system reasonably 




1 


1 






easy to use? 


. ' * 
\ i 1 


T 


2. 


Is it a rigid, step-by-step system? 


»' , . . J 


R 


3/ 


Are special skills or extensive preparation 




• ( 
1 


A 




required before the teacher can employ the 




1 
1 


N 




system? 




1 


s 


■4. 


If equipment is included, is it easily used 






1 


A 




by the teacher and by the students? 


i ! 


G 


5., 


Are. the system's materials readily used by 


1 J 1 


T 


• 


the students? 


' . ■ ( 


I 


6. 


Do you know of evidence of the system's 


< ■ • ' 






successful use in other educational ^ 


: i 


N 


• 


settings? ' 


1 ; 1 


s 


. 7. 


Does the system present a ny^. unusual storage 


1 ■ ' • • i 






problems or problems of distribution to \ 








students? 


! 1 

1 , ' , ! 



* 



6^ 









Yes 


No 


Don ' t 
Know 






B. MANNER OF USE--cont1nued 








1 


T 


8. Does using the system actively Involve 
the students? 

9. Does the system offer adequate evaluation 
procedures for the teacher or for the student? 

, vj • 19 tncrc a yuuu iiiatwri uctwccn tnc c/olUdulun 

procedures and the systems objectives? 
11. Did you find the evaluation procedures 
adequate? 






• 


/ 

\ 


R 
A 
N 
S 
A 
C 
T 
I 

0 . 

N 

S 


C. QUALITY OF CONTENT 








$ 

• 


1. Is the system's content authentic and (If 
applicable) accurate? 

2. Is the content tlWly (If applicable)? 

3. Are the Illustrations and examples apt for 
your students? , 

4. Does the content conmuntcate effectively? 

5. Are the system's sensory aspects (sound, 
colour, visuals, etc.) appropriate? 

6. Is the system designed for ease of 
student progression? 

1. Is the system's vocabulary appropriate for 

8. Is the content presented clearly? 

9. Did your students find the content relevant 
and Interesting? 




1 






0 
U 
T 

0 
M 
E 
S 


D. QUALITY OF CONSTRUCTION 

1. Does the system present any unusual problems? 

2. Did you fli^ the system sufficiently durable? 

3. Are the system's elements well designed? 

4. Did the system present any safety problems? 

5. If the system contains equipment, does it 
operate reliably? 

6. If eouloment reoalrs were needed, was the 
repair service adequate? 

7. Were such repairs required frequently? 

8. Were such repairs made easily and quickly? 


! 

i 

1 

\ 

1 


1 
1 
1 

1 
1 

" i 
» 

i 

\ 

, • f 


1 
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, i ^ Content Evali/atlon Questions 

(The following Is a list of questions addressed to trainees to be used in 
evaluating the content and procedures employed In a competency-based program.) 

Source: ^chey, R. C, 4 Cook, F. S. A comprehensive assessment and evaluation 
model for CBTE programs. Paper presented at the annual meeting of the American 
Educational Research Association, Washington, O.C., March 1975. (ERIC Documents 
Reproduction Service No. ED 117 176) / 

/ ■ '■•;./ 

1. Did prerequisite courses, if any, give you an adequate foundation in meeting 
the objectives of this course? 

/ ■ 

Yes No No Prerequisites 



2. Were adequate time, materials, and facilities provided for you to achieve 
this objective?! ; 

Yes No ; 



How meaningful were the assignments In relation to the mastery of this 
objective? ■ 



Asignments were Assignment helped Could have mastered Uncertain 

, helpful in ' some, but! could be objective without 

attaining mastery improved i completing asslgn- 

I ments ' 

4. How appropriate were the methods ^klts, films, etc.) used In conveying the 
Instruction to master this objective? 

Appropriate-- Were OK, but I Were Uncertain 

I liked them prefer other\ appropriate 

methods 

5. Did the exit test accurately measure ^he behavior sought in this objective? 
\ Yes No . 

6. ; Do you feel you have really mastered th\s objective? 

Yes, I feel very , Yes, but need some Yes, but need Did not pass 

competent 1 reinforcement \ much more exit test 

instruction 

7. In hindsight, do you think you could have bassed the exemption test on this 
objective at the beginning of the quarter w^lthout receiving Instruction? 

Yes No \ 

8. Did you feel this objective Is essential to your teaching preparation? 

'. / 

Yes No ; 
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Evaluation of Curriculum Design 



(The five components of this Instrument address relevant Issues of curriculum 
design: objectives, organization of materials, methods of instruction, evaluation, 
and total rating of the materials. It is useful in specifying Intended antece* 
dents for programs and products and for making judgments about the status of 
antecedents.) 



Source: Eash, M. J., Talmage, H., & Walberg, H. J. Evaluation of instructional 
materials . Mimeographed. Princeton, N. J.: ERIC Clearinghouse on Tests, Measure- 
ment, and Evaluation, 1975. (ERIC Document Reproduction Service No. ED 117 189) 

. \ 



I. Objectives - 

A. What is the nature of the general goals of the materials stated? 

B. Are specific objectives stated for teacher use? 

C. If neither of the above are stated, list what you believe are 
the Intended objectives of the material. 

D. What are the main emphases in the objectives? 

E. On the scale below, rate the objectives of the materials. Please 
place an "X" on an exact point. 

Objectives not use- Objectives give clear direction 

ful to a teacher 1 2 3 ^4 5 ^ for instruction and useful for - 

a teacher. 



II. Organization of Materials (Scope and Sequence) 

A. What is the scope of content covered in the materials? 

B. How is the scope of the materials organized? 

C. Is there a specified sequence In the material? 

D. What is the basis for the suggested sequence? f 

E. On the scale below, rate the scope and sequence of the material. S | 
Please place an "X" on an exact point. • 

Scope Inadequate, Scope adequte for grade in group, 

sequence not i':;,ical 1 2 3 3 5 6 sequence tasks carefully inter- 
or Incomplete related and planned. 

i 

i 

i 



* 
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Evaluation of Curriculum Design- -Continued 



III. Methods of Instruction 

A. What method or methods of instruction are suggested? 

B. What role Is emphasized in the method: teacher, pupil, or both? 

C. What are the specific features of the method. or methods recommended? 
0. Is the suggested method one that requires the^teacher to do extensive 

prior preparation or participate In specific training? 
E. On the scale below, rate the methods of Instruction. Please place an 
"X" on an exact point. | 



No methods suggested or Very carefully developed method 

implied that are helpful"! 2 3 4 5 5~ Very useful to both teacher and 
to a teacher pupil . 



rv. Evaluation 

A. What test materials are Included for the student's and teacher's use? 

B. Are the test items adequate for informing a teacher of students' 
progress toward the instructional objectives set for the materials? 

C. What do the tests measure? 

0. Is there information on the tests' reliability and validity? 

E. Is there any information from the producer on how the materials were 
tested with studients when they were being developed? 

F. On the scale below, rate the evaluation components of these materials 
and the evaluation of the materials by the producer as- they were 
developed. Please place an "X" 6n an exact point. 



No test materials or suggested A wide range of test 
checks on student learning -j — n — 3 — j — g — g- materials and evaluation 
included. No data on the eval- suggestions. Evaluation 
uation of materials by the data on field test con- 
producer, ducted and materials 



V. Total Rating of the Material 

A. Draw up a brief statement on how these materials compare with those 
currently being used in your curriculum. 

B. On the scale below, rate overall potential effectiveness of these 
materials. Please place an "X" on an exact point. 



included. 



Materials contain many 
weaknesses in instructional 
design. Difficult to use, 
expensive, inferior for 
learning. 



1 2 3 4 5 6 



Very strong in all areas of 
design. Strong potential to 
develop a wide variety of 



learnings. Of high Interest 
to teachers and pupils. Very 
cost effective. 
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Ipstructlonal Materials Evaluation Form «^ 

(The following Instrument for observing antecedents and transactions Is 
designed for curriculum-materials evaluation. With some modifications, an 
Instrument like this could also be used to assess programs. Judgments can 
be made on the basis of the Individual teacher's priorities and needs.) 

Adapted from McLaughlin, J. A..& Triica, J. S. Teacher evaluation of Instructional 
materials. Educational Technology , March 1976, 1^, 51-54. 



ANTECEDENTS 










T 




N/A 


The materlalS" 


I. 1. 


□ 


n 
kj 


□ 


rit cxistiny tcrnii na i od j cc u i ves . 


2. 


D 


n 


n 


van Dc cx tcnjOcu wO o vner pnases ot cum cu i uni. 


3. 


□ 


n 


n 


ric 1 p dccoinp 1 1 sn tnc udjcc ti vca vT tnc c^* ^ ' ' uni • 


4. 


□ 


□ 


□ 


Arc organizea Tor sequential ueveiopinenv-OT concepvS/SKi i is* 


5. 


□ 


n 

LJ 


n 


CT'm a 1% 1 a c ^ 1 1^ AM ^c am«sa^4#^a s a' a ■•is4a^s4a aamaaa^c/cI«4i 1c 

tnaoie siuaents tO practice ana rnainvain concepts/SKi i is. 


6. 


□ 


n 

LJ 


□ 


ftJsl^A MM«Mti4^4AM AtiAlllA^4AA M MA #4 MAC ^ 

naxe provision Tor evaiuaving progress. 


7. 


□ 


n 

Li 


n 


penmc student exploration* proDiem soivingt discovery. 


8. 




□ 


n 
U 


All At.a ^lAk^4^414^k« AmA 4 ^A ^AM 4M^4&i 4^11*1 ^4 ^^A mama A«* 

Allow TiexiDiiity and provide ror individual diTrerences. 


9. 


□ 


n 

LJ 


□ 


Allow independent u$e oy students. 


10. 


□ 


n 


□ 


Ua litftA^ L.«4 4U W41 4mAI|'«1 aU41^MAM 

tan be used with di lingual children. 


11. 


□ 


n 


n 

LJ 


A M» A ^ a A ^ 4 M A ^ 4*^1 1 A Am ^ tf* 

Are motivating to students. 


12. 




n 


□ 


Are llkelv to Interest the students. 


13. 


□ 


n 


U 


natcn tne reauiny level OT tne S tUQeil tS . 


14. 


□ 


□ 


□ 


Match the vocabulary level of the students. 


15. 


n 


□ 


□ 


Are multl -sensory In approach. 


16. 


□ 


n 


□ 


Contain graphic Illustrations. 


17. 


n 


n 


□ 


Contain clear, concise, easily understood Instructions 










for students. 


18. 


0 


□ • 


□ 


Contain easily followed, appropriate Instructions for 






n 




the teacher. 


19. 


□ 


□ 


Are presented In an appropriate, attractive format. 


20. 


n 


n 


n 


Are presented In an appropriate type size and material. 


21. 


n 


□ 


n 


Are of convenient, appropriate size ^nd number of parts. 


22. 


□ 


n 


□ 


Are of durable construction. 


23. 


• □ 


n 


□ 


Can be stored and moved, as needed. * 


24. 


0 


□ , 


□ 


Require In-service training for users. 


25. 


a 




□ _ 


Require adaptation for present purposes. 


26. 


a 


□ 


□ 


Justify their cost by their teaching value. 
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Package Assessment Scale (PAS) 

(The Package Assessment Scale (PAS), presented below. Is designed to pravlde 
evaluators with specific criteria for assessing the completeness and quality of 
Instructional materials. It may be used as an Instrument for formative or summa- 
tlve evaluation. Modified, the Items are also applicable to program evaluation. 

The authors suggest that the evaluators use their own response format for 
the Instrument (yes/no, or scale responses) and add their own Items. Decisions 
are to be made on the basis Of the needs dnd priorities of a given situation. 

Not Included In the Package Assessment Scale are Items relating to extra- 
instructional situations and activities.) 



Source: Hecht, A. R., & Klasek, K. R. PAS: A tool for developing or selecting 
self-Instructional materials. Audiovisual Instruction, April 1975, 20, 27-29,. 



A. Package Components and Sequence 

1. Does the package Include the following components: 

a. a rationale? 

b. cognitive learner objectives? * 

and 

affective learner objectives? 

or 

psychomotor learner objectives? 

c. a diagnostic pre-assessment? 

d. Interactive learning activities? 

e. periodic self-'assessment(s)? 

f. a posttest? . 

g. package development feedback? , 

2. Is the sequence of package components logical? 

B. Analysis of P ackage Components 

1. Does the rationale tell: 

a. what Is going to be learned? 

b. why this package should be studied? . 

2. Are the learner objectives: 

a. Important for intended learners? 

b. stated In terms which Include: 

1) an active verb Identifying desired behavior? 

2) content to which the behavior applies? _____ 

3) conditions under which behavior will occur? ~_ 

4) minimum performance standards? 
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3. 



Does the pre-assessment: 



a. measure learner status on each objective? 

b. have clear directions for completing and scoring? 

c. direct students to appropriate learning activities? 

4. Oo the learning activities: 

a. contribute to the achievement of package objectives? 

b. Include clear directions? 

c. pV'oceed In logical steps? 

d. occur In steps df proper size for Intended learners? 

e. attract and maintain student Interest? 

f. require active responses by learners? 

g. provide alternate learning activities for 
achieving each objective? 

h. employ media appropriate to package objectives? 

5. Do 'the learner self-assessments: 

a. Include clear directions for completing and scoring? 

b. provide frequent opportunity for learner self- 
assessment? 

c. direct students to appropriate learning activities? 

6. Does the posttest: 

a. Include clear directions for completing and scoring? 

b. measure learner status on each objective? 

c. parallel the pretesit form, length and difficulty 
(If a pretest Is present)? 

7. Does the package development feedback: 

a. focus on Important package characteristics? ^ 

b. Inlude objective and ess aV questions? 

C. Package Installation and Operation | 

1. Does the package Include an Instructor's guide? 
If yes. does the Instructor's guide: 

a. suggest ways Instructors can use the package 
with students? V 

b. suggest ways to help students schedule their 
learning time? 

c. suggest ways to record and report student progress? 

d. Include evidence of package revision on the basis 
of user feedback? 

e. Include evidence that appropriate package users 
achieve package objectives? 
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2. Are the equipment requirements of the package 
consistent with available audiovisual equipment? 

3. Is media quality •satisfactory? 

4. Are package materials easy for learners to use? 

5. Is the cost of Installing and operating the package 
within the limits of available budget? 

6. Is the time required for package Installation and 
operation within the limits of local schedule and 
calendar? 
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Checklist for Determining the Extent to Whicti a 
Program Is Competency-Based 

(The following is a detailed checklist that may be used by teachers to evaluate 
« program at any stage of CBE Implementation. . It follows the component outline of • 
PACIT and addresses pertinent Issues of CBE program planning and delivery.) 



Source: CDS Project 



Qoil Setting 

.1 . Students have an opportunity to consider all relevant 
information about the occupation, the training 
prograno. and themselves before making a final goal 

choice. 

2. Students are ablei to reset their goals in accordance 
with changes in their needs. 

3. Students can select goals from among a job ladder 
<such as nurse's aide or licensed practical nurse) that 
is consistent with the needs of the job market. 

4 Students can select goals from among several 
parallel specialties (such as brake specialist or 
tunf3-up specialist) tha^ are consistent with the heeds 
o( the job market 




Objectives 

1 Program is based on competencies derived from the 
lob through job task analysis. 

2 Conipetencies are specified in terms of measureable 
oh|e( lives which delineate the knowledge, skills, and 
attitudes required on the job. 

3 Competencies and objectives are specified in 
advance for each student. 

4 Each student learns only those competencies and 
objectives required to achieve their occupational 
goals. 

5. Each student's competencies and objectives are 
sequenced according to their goals. i 

6. Each student's objectives are'^equenced according to 
their instructional needs. 

7. Competencies and objectives are revised to meet the 
needs of students and the job market 



This is characteristic of 

3 

a. all of , b. parts of . c. norreof 
my program my program my program 



a. 



a.. 



a.. 



b.. 



b.. 



c. 



c. 



o 
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• 

r 

* 




This is Characteristic of 




a all of b. parts of 
my program my program 


c. none of 
my program 


Critcrion-Rtftrtnctd TMting 


-7- • 


• 




1 Standards for assessing students' mastery of compe- 
tertcies and obiectives are determined by the 
requirements iOf the job. 

* 


• 

a. 


h 


c. 








2 Students are assessed on only those competencies 
and objectives related to their occupational goals. 


a 


• 

c. 








3 Each student is given an opportunity to demonstrate 
mastery of competencies and objectives prior to 
instruction. 


a. 


• 

_ . h 


c. 








4 Assessment procedures allow learning experiences 
to be prescribed On the basis of each student's-needs. 






f- 




Learning Exptrionctt 








1 Individualized learning activities are used. 














2 Learning experiences use a variety of media and 
materials to meet individual student needs. 


a ... 


h . _ . 








— - 


3. Students are given enough time to master each 
objective rather than having to work within fixed time 
frames. 


a. 


h 


• 

1 

c. 








4. Learning experiences provide practice of skills in 
situations similar to those required on the job. 


a. 




c : 








5 Within each learning experience, provisions are 
madefpr letting the students Know how welf they are 

doing. 


a. ^ 




c _ 








6. Experts from the community (businessmen, lawyers, 
etc ) are used for the enhancement of learning 
experiences. 

7. Learning experiences use a variety of grouping and 
tutorina strateaies to meet individual student needs. 


a. 


h , .. 


1 

c. 


a. 


h 


ft 








A MniiiilAA nr iMrniM Activity oackaoM arfi uSdd to 
allow for flexibility in meeting students' needs. 


a. 


h 


c. 








9 Learning experiences are adapted when necessary to 








meet individual student needs. 
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This is characteristic of 



a. all of b. parts of c none of 
jny program my program my program 



Evaluation 






• 


1. Student assessment information is used to evaluate 
and revise learning experiences. 


a. . ^ 


h 


* 








2 Students* suggestions, comments, and observed 
behdvior^ are used to evaluate and revise learning 
experiences. 


a 












3. Student follow up information is used to evaluate and 
revise learning experiences. 


a.... 


h 


, c. _ 


Instructional Managtmont 






• 


f . Instructional materials and resources are organized 
to allow open-entry, open-exit education. 

2 Instructional materials and resources are organized 
to allow varied student pacing. 

3 Instructional materials and resources are organized 
to allow students a role in managing their own 
learning. 

4. Students, teacher aides, community volunteers, and 
other non-teaching personnel are involved in 
managing instructional materials and resources. 


a. 






a.. . 


b 




a. . 




C, 


a. 


b. 


c. 








5. Computer technology is used in managing instruction 
(such as keeping records, assigning learning 
experiences. Iceeping inventory of resources). 

• 


a. 




c. 


Otntral 




• 




1 Students are held accountable for their own learnlcig. 

i 

2 Emphasis >^s placed on exit requirements not on 
entrance requirements. 

3 Students take part in program decision-making (such^ 
as deciding on objectives, learning pace, learning 
experiences, etc.). 

4. Team approach to teaching is used to enhance 

learning. : * 

5 The systems approach involving planning, > 
monitoring, evaluating, and revising instruction is 
used 
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a. 


b 


c. 


a 


^ h 


f 


a. 


b 




a. 




* 

— c. 


a 


h 











Some It^ms to Observe When Judging Whether a Program Is Competency Based 



(The list of Items presented here describes activities or products that are 
characteristic of competency-based programs. It can serve as 'a checklist for 
deciding at which stage of CBE Implementation a program Is functioning and for 
det<»nn1n1ng the types of activities or products yet to be developed. 

The CBE program coiiponents In this list follow the component outline of the 
FACIT training system.) 



Source: CDS Project 



\ 


Pertaining to the Program 


Pertaining to the Students 


\g ^ 

\ 

E ' 

T 

f 

■ 'A 

x 


Sources for obtaining job-market 
Information and Information about 
job requirements have been Identi- 
fied. 

Job- related Information has been 
obtained and documented. 


Student-related Information has 
been obtained to determine 
«fhether a student's goal Is 
appropriate. 

Ways to deliver goal rse tiding ' 
Information to students have 
been developed. 

Students are being assisted In 
their goal -setting activities. 

There are ways to help students 
solve their goal -setting 
problems. . ' 


\ 

0 

B ^ 
J 

E 
C 
T 
I 

V , 

E 

S 


Unit-level objectives are avail- 
able for the program. ' 

\ Unit- level object Ives -have been 
documented and sequenced. 

Objectives have been validated. 

Hhere necessary, unlt-fevel 
objectives have been broken 
down Into subobjectlves. 


Objectives are being selected 
to match students' goals. 

Objectives are being selected to 
meet Indlvldu^ learner needs. 

Objectives are being sequenced' < 
' to meet; Individual learner needs 
and preferences. ' [ 

• • 
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. V ■ . • J 

« 

4 


• 


Pertaining to the Program 


.-"^ Pertaining to the Student^ 


R 

C E 
R F T 
I E E 
T. R S 
E E L ' 
R N- I 
I C N. 
0 E G 
NO. 


Criterion-referenced tests have been 
obtained or prepared for each objec- 
tive. 

Criterion-referenced tests reflect 
the requirements stated In the 
objectives. 

Criterion-referenced tests have 
been evaluated and found to, be 
appropriate. 


Criterion-referenced pretests are 
being administered to the students 
when appropriate. . '1 / 

* / 

' Criterion-referenced postte&ts are * 
be1ng*^dm1n1stered to students 
foil owing, the ccnpletlon oj 
learning experiences for each 
objective. / 

Administration of criterlon- 
" referenced tests Is modified to 
meet learner needs. 


E 

L X 
E P 
A E 
R R 
N t 

*' N N 
•G C 
E . 
S 


^Learning experiences are available 
for each objective. 

• 

Alternative learning experiences 
are available for each objective. 

Instructional materials are 
appropriate for the learners and 
for the tasks to be taught. . ^ 


• s 

Instructional strategies ahd media 
are selectedi^to match student'^ 
characteristics. ' ' ' 

Each student Is able to proceed at 
at a pace appropriate for him or her. 

Tutors are used to help Individual 
students. 

Grouping Is used where appropriate. 


E 
V 
A 

> L 
U 
A 
T 

N 


There are ways to determine which 
learning experiences are In^ need 
ofj evaluation. 

Data collection devices and 
procedures have been decided 
on and are being, used. \ 

\ 

Instruction Is being evaluated 
and revisions are mad6 w^ere " 
necessary. 


Students are encQUraged tomovld^ } 
feedback on learning- expeivences. 

^ I / ■• ' 

• Student records afe^x'amffofid 
per1od1ca.lly to determine 
problems with Instruction. ' , . , 

Student follow-up data ^r*e collected 
and used as a basis for revising^ ^ 
Instruction. 


I 

N M 
S A 
T N 
R A 
U G 

• ; C E 

T H 

- I E 
O.vN , 
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An individual student record 
system that centers around 
objectives^ used. 

Class progress records that 
center around objectives are> 
belnti k^Dt iA 

A system to monitor resources 
has been established. 


' J ' — ■ ■ ■ ■ 

Students have access to their files 
in order to monitor *t heir own , / 
progress. 

Students have access to.i^sources 
when needed. . . 

Prpctors and aides are used to 
facilitate classroom processes. 

/ 

1 • - 





Pertaining to the Program 


Pertaining* to Students 


I 

N M 

S A C 
T N 0 
RAN 
U G T 
C E I 
T M N 
I E U 
0 N E 
N T D 
A 
L 


Files that permit easy access and 
, easy monitoring have been set up. 

Aides and proctors are used to 
monitor the management system and 
keep It up to date. 

* • 


A grading system that reflects 
the student's status in relation 
to performance on objectives is 
used . 

* 

• 

• 
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Product Checklist for FACIT 



(The following is a checklist of products that should result from carrying 
out the activities suggested in FACIT or using the skills taught through FACIT. 
It follows the component outline of the FACIt training system and lists products 
by FACIT competency.) 



Source: COS Project 

GOAL SETTING 

Competency A; Identifying Types and Source^ of Goal -Setting Information 

1. List of job-information sources 

2. Information brochure containing-- 

a. local market information and Job outlook 

b. physical, academic, personal characteristics needed for job 

c. job characteristics 

d. training requirements 

3. Sources of student information 

.4.- Student profiles that include-- 

a . needs 

b. interests 

c. job characteristjcs 

d. training requirements 

Competency B; Helping Students Solve Goal-Setting Problems 

1. Plan for diagnosing student problems 

2. Plan for helping students in re-establishing goals 



OBJECTIVES 

Competency A: Selecting and Sequencing Objectives 

1. A sequence of objectives for each student 

Competency B; Analyzing Objectives 

1. For each unit objective, a list of the necessary knowledge and skills 
that the student will be expected to master 

Competency C; Writing Objectives 

1. Instructional ly complete objectives (unit-level obj^tives and enablers) 
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Competency D; Validating Objectives 
1. A plan for validating objectives 



CRITERION-REFERENCED TESTING \ 
Competency A: Using Criterion-Referenced Pretests and Postt^ts 

1. A plan for determining when It Is appropriate to pretest o student 

2. A plan for making appropriate Instructional decisions an the basis 
of a student's pretest performance 

3. A plan for making appropriate Instructional decisions on the basis 
• of a student's posttest performance 

Competency B; Determining Types of Tests to Use 

1. A for detennlning the appropriate kind of tests to use In 
measuring mastery of each objective 

Competency C; Identifying Test1n»-^u1rements of Objectives 

1. A plan for determining the criterion for each test to be given 

2. A plan for determining the number of items to include on wri^en tests 

i 

I 3. A plan for determining ;he number of times performance shoul<^ be 
demonstrated on ski V. tests 

Competency u: Developihg Pc forman ce -Rating Methods 

1. Checklists 

2. Rating scales 

Competency E: Constructing Selection Items 

1. Multiple-choice tests 

2. True- false tests 

3. Matching tests : 
Competency F: Constructing i -^ ply Items 

1. Completion tests 
Z. Short-answer tests 
^3. Essay tests 
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LEARNING EXPERIENCES , 

Cornpetencies A> B. C. D; Pla nning Learning Ex periences Using the F unctions of 
I nstruc tion' as* a Planning Frameworlt 

1. Lesson plans for each objective 

2. Plans for complete learning experiences for the students 
Competency E: Choosing Media for Instruction 

1. A list of available media 

2. Plans for how and when different types of media will be used 
Competency F: Rev i e wing and Selecting Instructional Materials 

1. A file of materials to be used 

2. Materials organized to permit easy accf»ss by the students 
^ Competency G: Adapting Instructional Materials 

1. Instructional materials, modified to meet the requirements of the 
student population 

2. For each objectivp learning packages delineating products and activities 
Competency H; Individualized Pacing 

1. Plans for individualized pacing 
Co mpetency I: Tutoring 

1. Plans for obtaining, training, using, and monitoring tutors 
Competency J : Flexible Grouping 

1. Management procedures for grouping 

.EVALUATION 

Compete ncy A: Evaluating Learning Experiences ' 

1. A plan for collecting and analyzing information in order to evaluate 
learning expe-lences * 

Competency B; Obtaining and ^Using F^ol low-up Information 

\1. A plan for collecting and using follow-up information in order to determine 
the effectiveness of learning experiences and the appropriateness of 
objectives 
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INSTRUCTIONAL MANAGEMENT 
Competency A: Keeping Progress and Performance Records 

1. Records on Individual student progress and on overall class progress 

Competency B: Ordering and Monitoring Instructional Resources 

,1. A system for ordering appropriate quantities of consumable and non- 
consumable resources 

2. A system for monitoring use of consumables and nonconsumables 

3. A plan for using aides to perform routine tasks 

Competency C: Allocating Instructional Resources 

1. A system for managing resources so that students will be able to use 
resources when needed 

Competency D: Using Proctors and Aides 

1. A plan for selecting, training, using, and monitoring proctors and aides 

Competency E: Determining Grades' 

t 

1. A procedure for converting achievement and progress into letter grades 
Competency F: Organizing Records 

1. Instructor's file 

2. Student learning progress file 

3. ' Resource file 
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Question Areas for Data-Based Decision Making 



(The followlig provides some general questions about program antecedents, 
transactions, anc' outcon.as; these questions Indicate areas of Investigation 
related to objectives, resources, instructional strategies, and program costs.) 



Adapted from Hall, G. E., & Jones, h L. CBE program evaluation and data-based 
decision making. i In Conipei ^ncy-based education: A process for the improvement of 
education . Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1976. 





OBJECTIVES 


RESOURCES 


STRATEGIES 


COST 


A 
N 

T 
E 
C 
E 
D 
E 
N 

T 

1 

s 

T 
R 
A 

N 
S 

A ' 
C 
T 
I 

0" 

N 

S 


Why are we not get- 
ting what we want? 

What are the 
desired outcomes? 


What resources do 
we have? . 

What resources do 
we need? 


Which instructional 
strategy do we want 
to use? 

How should we 
Implement? 
(Which adoption 

strategy should 

we use? What 

training is 

reauired?) 

•» / 

\ 


How much is it 
costing to plan? 

• 

How much will it 
cost to Implement? 

Once implemented, 
what will main- 
tenance cost be? 

How long will 
materials last? 

How much staff 
time is needed to 
implement? 


Does it look as if 
^e are going to get 
the outcomes we 
want? 

What unexpected ' 
outcomes are we 
getting? 

C 


Exactly what 
resources are 
needed? 

Will -our resources 
support CBE on a 
regular basis? 


Are the instruc- 
tional strategies 
working? 

What changes must 
be made in the 
strategies? 

Is further faculty 
training needed? 

Will everyone be 
able to do his ; 
or her part? 


What is it costing 
to Implement? 

What do the main- 
tenance costs now 
look like? 


0 

u- 

T 
C 
0 
M 
E 
S 


« 

What outcomes -are' 
we regularly 
getting? . 

What un^expected 
outcomes are we 
getting? 

What new needed 
outcomes have been 
identified? 


What resources are 
now freed up? 

What unexpected 
resources are 
being consumed? 


How effective were 
the implementing 
strategies? 

^Hjow effective are 
the Instructional 
strategies? 


How much did it 
co^t to plan for 
and implement this 
innovation? 

How '^uch is it i 
costing to main- 
tain? 
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Competency- Based Program Evaluation Criteria 



(The chart following details criteria that may be applied In evaluating a 
professional training program that "Is competency based. ) 

Adapted from Houston, W. R., et al. Criteria for describing and assessing 
competency based programs. In W. R. Houstop (Ed*)» Competency assessment , 
research, and evaluation. A report of a national conference. University" of 
Houston, March 12-15, 1974 . Washington, D.C.: American Association of 
Colleges for Teacher Education, 1974. (See pages .169-171. ) 
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Competency Specification 



• Competency statements are 
specified and revised on the 
basis of an analysis of job 
definition and a theoretical 
formulation of professional 
responsibilities. 

• Competency statements describe 
outcomes expected from the 
performance of profession- 
related functions, or those 
knowledges, skills, and atti- 
tudes thought to be essential 
to the performance of those 
functions. 



• Competencies are treated as 
tentative predictors of pro- 
fessional effectiveness and 
subjected to continual vali- 
dation procedures. 

• Competencies are specified 
and made^ public prior to 
Instruction. 



Competency statements facili- 
tate criterion-referenced 
assessment. 

Learners completing the CBE 
program demonstrate a wide 
range of competency profiles, 



Instruction 



Assessment 



The instructional program is 
derived from and linked to 
specified competencies. 

Instruction that supports com- 
petency development is organ- 
ized into units o^ manageable 
size. 



Instruction is organized and 
constituted so as to accommo- 
date learner style, sequence 
preference, pacing, and 
perceived needs. 

The extent of learner's pro- 
gress in demonstrating com- 
petencies is made known to 
him or her throughout the 
program. 

• Instructional specifications 
, are reviewed and revised on 
the basis of learner feedback 
data. 



• Learner progress is determined 
by demonstrated competency. 



Competency measures are 
validly related to competency 
statements. 

Competency measures are spe- 
cific, realistic, and sensi- 
tive to nuance. 

Procedures for measuring com- 
petency demonstration assure 
quality and consistency. 

Competency measures allow for 
the Influence of variables 
in. setting upon performance. 



Assessment procedures and 
criteria are described and 
made public prior to instruc 
tion. 



Competency measOres discrimi 
nate on the basis of stan- 
dards set for competency 
demonstration. 

Data provided by competency ^ 
measures are manageable and 
uteful in decision making 



Governance and Management 



§ There are statements of policy 
that dictate In broad outline 
the Intended structure, con- 
tent, operation, and resource 
base of the program. Includ- 
ing the teaching competencies 
to be demonstrated for exit 
from the program. 

• Management functions, responsi- 
bilities, procedures, and 
mechanisms are clearly de-. 
fined and made explicit. 



• Formally recognized pro- 
cedures and mechanisms exist 

' for arriving at policy 
decisions. 

• 

• A formally recognized policy^ 
making (governing) body 
exists for the program. 

f All Institutions, agencies, 
organizations, and groups 
participating In the program 
are represented In policy 
decisions that affect the 
program. 



f Policy decisions are supported 

ty, and made after conslder- 
* atlon of, data on program 
effectiveness and resources 
required. 

• Management decisions reflect 
stated program philosophy 
and pol Icy. 



* Staff Development 



Provisions are made for staff 
orientation, assessment, and 
Improvement. 



Staff -development programs 
are based upon and engaged 
Inaftet consideration of 
data on staff performance. 



Program staff attempt to 
model the attitude's and 
behaviors desired of students 
In the program. 



Tttal Program 



They^irogram Is planned and 
opierated as a totally unified, 
Ihtegrated system. ( 



§ Research and dissemination 
activities are an Integral 
part of the total Instruc- 
•tlonal system. 

§ A research strategy for the . 
validation and rev1$1on of 
program components exists and 
Is 'operational . 

§ A data-based management system 
Is operational. 

§ Procedures -for systematic use 
of available data exist. 



• Institutional flexibility is 
sufficient for all aspecJts of 
the program. 

• Reward structure in the 
institution supports CBTE 
roles^and requirements.i^^ 

Financial structure (moSSes 
and other resources) in the 
system supports collaborative 
arrangements necessary for 
the program. 



A 
N 
T 
E 
C 
E 
0 
E 
N 
T 
S 



T 
R 
A 
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Governance and Management 



• The Identified professional 
with responsibility for 
decision has authority and 
resources to Implement the 
decision. 

• Program operations are 
designed to model the 
characteristics desired 
of schools! and classrooms 
In which {>rt)gram graduates 
will teach. * 

• Job^,def< nitrons, staff ^ 
seleHlpns, and responsi- 
bility assignments are 
linked to the management 
functions that exist. 

• There are formally recog- 
nized procedures and 
mechanisms for arriving at 
the various levels of pro- 
^gram-management decisions,. 



NA 



NA 



Staff Development 



NA 



Total Program 



NA 



NA 



NA 



NA 



, • Grading and program- revision 
procedures are flfexible. 



*Not applicable 



Measures That Can Be Employed for Program Description 

(The following is a summary of suggested measures for program description. 
The References for the orj^inal documents of which this list is? a sunmary will 
follow this summary outline.) • ■ ' t . 

Adapted from Sechrest, L. Use^of innocuous and non-interventional measures 
in evaluation. In B. R. Worthen & Y.'^R. Sanders Educational evaluation ; 
Theory and practice . Worthington, Ohio: Charles A. Jones Publishing Co., 
1973. (See pifles 283- 303 . ) 

A" ' • 

( • 

I. Antecedent measures ' , ' ■. ■ \ 

A.\jPupii background files ' / 

X. .Mental jabil.ity scores • • 

2; Pai^ aqhievement 

3. Sex • ! \ 

4. Father's occupation 
, 5. '^thnit background 



B. Community' records 

1. Ci.ty records* ( 

2. Chamber of commerce files 

3. Etc.'-^ 

I 

C. Questionnaire- techniques 

r. Attiiude measures 
2. DeiAo^raphic measures 

D. Measurements of cos^ 

1. Facilities 

2. Progi[am 

' 3. Personnel 

4. Student 

E. Teacher-t)ackgr6und files 
ing 



i 



1. Trair 

2. Experience 

3. Ethni 

4. Sex; 



C|fackg,round ^ 



^5., Length of time s^ent In jobs 



F. ; Curricular-conteixt dbscript-ion • ^ • 

1. Descriptive, characteristics (media, sources, time needed, style, 
cost,i'jayailability,r.user data, content, characteristics ojf curricul 

» ■■■/•' 



/ 

/ 



2. Rationale and objectives 

3. Required conditions (pupil, characteristics, teacher capabilities and 
requirements, community and school requirements, arttculatlon between 
program areas) o • 

4. Content characteristics (cognitive content, affective content, . 
psychomotor skills) ' 

5. Instructional theory and teaching strategies 

6. Overall assessment of curriculum (other descriptive data, effects 
reported ^predicted by these 'sources, recommended uses) 

II. Tra>isacjt1 on measures " " . 

* 

A. Currlculum-lmplementatlor^ description 

1. Objectives used (Impilcit and stated) 

2. Organization of materials 

3. Methodology employed v 

4. Evaluation techniques 

B. Environmental measures 

C. ^Interaction analysis 

III. 0 .''.confe measures 

A. Cost/benefit techniques ^ 

8. Sampling techniques 

C. Computer-assisted testing techniques 

D. Empirical scoring techniques ' _ j 

E. Sequential testing techniques / 

F. Scale construction techniques ' 



Original sources: 

ft 

Eash, M. J. Developing an Instrument for the assessment of Instructional 
materials (Form IV) . Paper presented at the amiual meeting of the American 
Educational Research Association, Minneapolis, Minn., March 1970. 

Morlsett, I., & Stevens, W. W. Steps In curriculum materials analysis outline . 
Mimeographed. Boulder, Colo.: Social Science Education Consortium, University 
of Colorado, 1967. 

Sjogren, D. C. Measurement techniques In evaluation. Review of Educational 
Research , 1970, 40. 301-320. 

Tyler» L. L. , » Uleln, M. F. Recommendations for curriculum and instructional 
materials . Mimeographed. Los Angeles: University of California at Los. Angeles, 
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Information Categories for -Evaluation Data Collection 

(The. following suggests some general data for making decisions about a 
program or product. Although not explicitly stated here, data suggested can 
and should be applied to Intents and observations.) 

•Adapted from Foley, Walter. The future of administration and educational 
evaluation. In Evaluation of education . The Educational Technology Review 
Series, No. 11. Englewood Cliffs, N. J.: Educational Technology Publications, 
1973. (See page 72.) 





ANTECEDENTS 

r1 111 k W k W k 1 1 1 W 






(Input) I. 


Systems variables 


II. 


Environmental constraints 


• 


A. Pupil data 

B. Staff data 

C. Facility data 

D. Financial data 

E. Curriculum data 

F. Other data , 


III. 


Input constraints 

A. Pupil -s^election criteria 

B. Staff-Selection criteria 

C. Other criteria 




TRANSACTIONS 






(Process) 


I. Organ lYational cycle variables 


II. 


Environmental contraints 




A. Pupil tasks and .interaction 

B. Staff tasks and Interaction 

C. Extra-setting tasks and 
interactions 

D. Other data 


III. 


Process constraints 




OUTCOMES 


0 




(Output) 


I. Termination variables 


II. 


Environmental constraints 




A. Pupil data 

B. Staff data 

C. Facility data 

D. Financial data 

E. Curriculum data 

F. Other data 


III. 


Output constraints 

A. Pupil objectives 

B. Staff objectives 

C. Other objectives 

• 



« 



Program Evaluation Informatlorv^ 

(The following is a list of the kinds of information that are potentially 
available for evaluating comeptency-based educational programs. The list Is 
organized according to the techniques. used to collect the. information. Some 
of the information listed relates to program antecedents, some to program trans- 
actions, some to program outcomes, and some to all three data-collection cate- 
gories.) 

Adapted f rom Metfessel , N. S., & Michael, W. B. A paradigm involving multiple 
criterion measures for the evaluation of the effectiveness of school programs. 
In B. R. Worthen and J. R. Sanders (Eds.), Educational evaluation; Theory and 
practice . Worthington, Ohio: Gharles A. Jones Publishing Co., 1973. 

1. Cognitive^and affective outcome data obtained from standardized Instruments 
0 Scores on achievement and ability tests 

0 Self-reports regarding attitudes, values, opinions. Interests, and so on 

0 Ratings of the quality of student products 

0 Assessments of psychomotor and physical -fitness skills 

2. Cognitive' and affective outcome data obtained from teacher-made Instruments 

0 Student responses to Incomplete sentences 

0 Student interview re$ponses 

0 Peer nominations and recownendations 

0 Student questionnaire responses 

0 Self -concept information 

0 Self-evaluations of achievement, adjustment, etc. 

0 Attitude information derived from role playing, picture Interpretation, 
and so on 

0 Scores on teacher-made achievement tests 

0 Teacher observations of classroom behavior 

3. Behavioral outcome data of a general Tiature obtained by nontest means: 

0 Student absences and tardiness 

0 Records of critical incidents 

0 Records of appointments kept or broken 

p Information on assignments completed 

b Autobiographical information 

0 Awards or other Indications of significant achievement 

0 Case 'listories including significant events in students' lives 

0 Students' requests for changes in program or teacher 

0 Students' educational, vocatinal, or leisure-time choices 

0 Student interactions 

0 Disciplinary actions 

0 Dropout rate 

0 Elected positions held 
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0 Participation In extracurricular activities 

0 Grade-placement Information 

0 Grade-point average 

0 Student grouping patterns 

0 Completion of homework assignments 

0 Pursuit of leisure activities 

0 Use of library 

0 Student course Toad 

0 Peer-group participation * 

0 An^l -social and asocial student behaviors 

0 Referrals by counselors, psychologists, etc. 

0 Student self-referrals 

0 Social mobility 

0 Student transfers 

Behaviors of teachers and other school personnel determined by n ontest means; 
0 ' Publ Ished articles • 

0 Attendance at professlonol meetings, summer school, continuing education 

classes, etc. ' 
0 Elective offices 

0 Grade-point average In postgraduate courses 
0 Student- teacher and student-counselor ratio 

0 Correspondence about teachers, counselors, administrators, and other 
school personnel 

0 ' Memberst)1p» In professional organizations - . 
0 Congruence between program-Implementation requirements and the behavior 

of program personnel 
0 Outside Jobs held by school' personnel 
0 Nominations for outstanding service 
0 Ratings of professional competence, skills, and attitudes 
0 Staff resignations and' dismissals 
0 Staff transfers 

Relevant community behavior determined by nontest means ; 
0 Alumni participation In school activities, 

0 Public attendance at schonl events, school board meetings, and so on 
0 Parent-teachef , parent-counselor, and parent-administrator conferences 
requested by school 

0 Correspondence from parents and other communUy members regarding school 

evppts. outcomes, etc. 
0 Parental responses to student grades, behavior, and so on 
0 Telephone calls from parents and other conmunity members regarding school 

events, outcomes, etc. 
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Ways by Which Program-Description Information Can Be Reported 



(The foUGwing shows some reporting techniques for documenting data relevant 
to a competency-based program.) 

Adapted from Richey, R. C, & Cook, P. S. • A comprehensive assessment and evaluation 
model for CBT E programs . Paper presented at the annual meeting of the Americar*/ 
Educational Research Association, Washington, D. C, March 1975. (ERIC Document 
Reproduction Service No. ED 117 176) 

* • 

1. S tatus Reports, whjch include-- 

a. a list of objectives for a class • 

b. number of students passing objectives 

c. date each passed 

d. number of students repeating an objective 

2. Histograms , which include-- 

a. number of students passing the exemption tests 

b. number of students passing the exit tests 

c. time required to complete each objective 

d. number of students repeating an objective 

3. Curriculum status reports, which include— 

■ 

a. a list of students in the curriculum 'area 

b. a list of all objectives completed 
c: date of completion 

d. a list of objectives not completed 

e. number of students repeating an objective 

f. recorded student problems 

. . 

^' Student status letters , which include-- » 

a. a list of all objectives to date 

b. a list of objectives completed 

d. dates of completion /* 

e. special message indicating next steps 



93 ^ 0 



Informition Collection for Transactions 
^ Occurring in a CBTE Program 

(The questions below provide ideas for the types of data that can be used 
to assess specific transactions occurring in a competency^based program. Ante- 
cedents and outcomes for intents and observations are not addressed specifically 
in this list. 

In addition to the questions themselves, sources and techniques for obtaining 
input that will answer the evaluation questions are listed.) 



Adapted from Loucks, S. F.- Exhibit 10.11: Evaluation questions. In G. E. Hall 
& H. L. Jone,Sj Competency-ba sed education: A process for the improvement of 
educatio n . Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1976. (See paqes 
290-294,7 ' V H y 



QUESTIONS 


SUBJECTS 


KIND OF 
INSTRUMENT 


t 

I. FOR THE TOTAL PROGRAM, 
WHAT ARE POTENTIAL SOURCES 
OF FAILURE? 






A. What is the quality of inter- 
personal relationships among 
staff and students? 


Faculty 
Students 

Cooperating teachers 


Questionnaire 


B. Are communication channels 
adequate? 


Facul ty 
Students 

Cooperating teachers 


Questionnaire 


1. Do you always know what 
is expected of you? 

2. Do you know what other 
team members and cooper- 
ating teachers are doing 
with the same students? 

3. Do you know what students 
are doing in class? 


Students 
faculty 

Cooperating teachers 


Questionnaire 
Questionnaire 

Questionnaire 

- _ - ... -J 
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QUESTIONS - 



C Are resources adequate? 



D. Are morQ needed? 



1. What materi/dls are being 
used? 



2. What materials are not 
beipg used? ^ 



SUBJECTS 



Faculty 
Students 

Cooperating teachers 
LRC staff 



Faculty 
. Students 

Cooperating teachers 
LRC staff 

LRC staff 
. LRC 



KIND OF 
INSTRUMENT 



Questionnaire 



Questionnaire 



Questionnaire 
Materials use list 

Questionnaire 
Materials use list 



E. Are physical facilities 
adequate? 

1. Are designated ones being 
used? 



Faculty 
Students 

Faculty 
Students 



Questionnaire 
Observation* 

Questionnaire 
Observation* 



Is 


staff adequate? 


Facul ty 






Students 






Cooperating 


1. 


Are there enough faculty 


Faculty 




members to meet student 


Students 




needs? 


Cooperating 


2. 


Do staff members provide 


Faculty 




good models? 


Students ' 




Cooperating 


3. 


Are staff members acting 


Students ' 



as guides, facilitators, 
discussion leaders rather 
than information givers? 



Questionnaire 



Questionnaire 



Questionnai re 



Questionnaire 



G. Is time schedule adequate? 



1. How are staff spending 
time? 



2. How are students spending 
time? 



Faculty 
Students 

Cooperating teachers 
Faculty 

Cooperating teachers 

Students 
1^-^^ 



Questionnai re 



End-of-day reaction 
sheets given random 
days once a week 



I 



QUESTIONS 



H. Are logistics being handled 
adequately? 



1. Are materials available 
when needed? 

2. Are organizational arrange- 
ments always worked out. in 
advance? 

3. Is LRC meeting needs? 



SUBJECTS 



Faculty 

Students --^ 
Cooperating teachers 
LRC staff 



I. Are participants satisfied 
in general? 



Faculty 
Students 

Cooperating teachers 



KIND OF 
INSTRUMENT 



Questionnaire 



Open-ended question- 
naire given once a 
week 



J, What specific prq^lems 
are perceived? 



Faculty 
Students 

Cooperating teachers 



Open-ended question- 
naire given once a 
week 



II. ARE PROGRAM COMPONENTS BEING 
IMPLEMENTED? 

A. Have the competencies been 
stated? 

1 , ^Are they public? , 



/ 



Faculty 
Students 



Interview 
Interview 



B. Do the competencies determine 
instriKjtion? 



Foculty 



Interview 



C. Is the student assessed on 
attainment of competencies? 



Students 



D, Are modules being used for^ 
instruction? 



E. Are students using modules? 



Students 
LCR staff 



F. Do faculty share the same 
students? 



Faculty 



Interview 



Observation** 

/ 



Interview 
Observation** 
Materials use list 



Interview 
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QUESTIONS 


SUBJECTS 


KIND OF . 
INSTRUMENT 


G. Does the team meet to share 
^ planning? 


Facul ty 


Interview 
Minutes of team 
Meetings 


H. Is student time blocked? 


Facul ty 
Students 


Interview. 


I. Is the time that is blocked 
used for instruction or 
related activity? 


Facul ty 
Students 

Cooperating teachers 


Observation 
End-of-day reactior 
sheets 

— —" — - — — — - * . 


J. Do the students spend the 
majority of time in the 
schools? • 


Students 


Observation* 


K. Is instrbCtion related to 
experiences in the classroom? 


Facul ty 
Students 


Observation* 


L. Is time spent with children? 


Students 


Observation* 


III. WHAT IS ACTUALLY TAKING PLACE? 


Facul ty 
btudents 

Cooperating teachers 
LRC staff 


Observation* 


* Observation in schools. 
** Observatior in seminars. 
*** Observation in LRC. 






t 


• 

\ 


/ 

0 
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Four-Phase Evaluation Model 



(The su9gestions below are an extension o^ the De/iton CBTE evaluation model 
to occupational tra'ining programs in genefal . What to assess is addressed, as 
well as how to use the information obtained through assessment in evaluting the 
training program. With respect to the categories of the Stake model, the Denton 
model is incomplete; it does not adcpress explicitly antecedents and transactions 
Nor does the Denton model deal with program implementation from a level s-of-use 
perspective. Its primary concern is with program effectiveness.) 

Adapted from Denton, J.J. A fi el "d- tested evaluation model to assess a CBTE 
program. Educational Technology , March 1977, U» 23-27. 



P hase One : Monito ring Content Acquisition 

l^. Data conection--Data collected during this phase enables the evaluator 
to describe the program primarily inoteps of /Cogniti^Ve and affecti-ve 
lOU/comes. 




Information Desired 



Metho.ds/Means 



a. Student achievement 



Student attitudes toward learning 
experiences 



a. Examination of* the number of 
students- mastering each 
objective, the numbep-of 
mastery attempts ,^ and so on.. 

b. Questionnaire and interview 
assessment of student opinions 
about the learning experiences. 



ERIC 



Data use--The program-description data collected during this phase is used 
to revise in-class learning experiences, especially any modules ,pr other 
materials used to teach the concepts and. theory of the occupational program. 



Evaluation data on learning 
materials: modules, textbooks, 
multi-media materials, manuals 
etc. 



D. Evaluation data on learning 
activities • 



h. 



Solicitation of student feed- 
back (informal comments or 
questionnaires) on the appro- 
priateness of learning materials 
ease of comprehension, organ- 
ization, media employed, 
practice exercises and tests, 
completeness Of the information 
presented, etc. * 

Solicitation of student feed- 
back on the appropriateness of 
the in-class activities: 
cccessibility of materials, < 
availability of supplies and 
equipment, supervision in 
practice activities, personal 
attention when explanations 
are needed, etc. 
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P hase Two: Knowledge- and Skill Assessment ' , 

1. Data colle^tion-.-Data. col l§cted during this phasp enable the evaluator 
to describe final cognitive outconies and sj:udent performance^ on hands- 
on skilly. ^ ' ' 



2. 



— y lnformation Desired 

a. Student achievement- in the theoreti- 
cal aspects of a course or program. 



b. Student performance on hands-on 
skflls acquired within the 
classroom situation. o 



Methods/Means • 
» . — . 

a. 'Posttest Qdta on- terminal ' 
objectives. Level of mastery 
achieved by the student^: 

b. Pejrformance ratings (on rating 
scales or performance check- 
lists) that have been made 

in in-classroom demonstration 
of specific ^skills using actual 
objects or simulations. 



Data use--Data col lectecj^ during this phase enable the evaluator to suggest 
or makfe revisions in objectives or presentation of content, or in the 
methods used in teaching specific occupational skills. 



Evaluative infomation on 
appropriateness of objectives, 



Evaluative information on 
students' performance of 
hands-on skills. 



a. Students' test performance 
on terminal cognitive objec- ^ 
tives. Observation of students 
in their application of 
knowledge -to practical problems. 
Review of objectives by con- 
tent experts or professionals 
in the field. 

b. Observation during 'demonstra- 
tion: documentation of skills 
or subskills that have not 
been acquired completely or 

\ that are regularly carried out 

\ incorrectly by students. 

Direct input from students on 
those skills for which they 
« feel th^ey require more practice. 

Phase Three: Student and Skill Assessment in a Real Setting 

1. Data collection— Data collected during this phase enable the evaluator to 
describe student outcomes as they rel^e to application of skills in a 
realistic situation. 



Student performance 
job situation. 



in an actual 



Observation of students during 
practicum work by practicum 
supervisdrs or the teacher, 
using rating scales or per- 
formance checklists. 



ERIC 
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I -. . ..... 

Data use--Data collectejf in this phase enable the evaluator, to judge 
wheth^ the skills acquired jn the classroom are transferred to the 
actual job situation.' = . 



Information Desired ' 

a. Evaluation input from practicum 
supervisors- (shop foremen, super- 
vising nurses, etc.) on student 
practicum performance. 

b. Evaluation input from teachers. 

\ 

C.. Evaluation input from students 
"on *'ieTd experience. 



" Methods/Means 

■ • 

^i. Interviews with on-the-job 
practicum supervisors on 
student performance or lacjc 
of skills. 

A 

b. ■ Ratings by teachers while 
they observe students during 
field experience. • " ' 

or~\ Interviews with students to 
determine their feelings about 
their Jevel of competence in* 
terms of applying skills 

- learned in the classroom to / 

- realistic situations. 



Phase Four:' Follow-up Assessment 

1. Data collection— Data collected during this phase permit the ^valuator 
to make descriptive statements about overall program effectiveness. 



Student performance on the jcb 
as viewed by the employer. 



a. Periodic interviews with 
employers about level of 
competence and number of 
skills that students demon- 
strate or lack. 



b. 



c. 



Student performance on the job 
as viewed by the former students 
themselves. 



Outside- observer impressions of 
former-student performance on 
the job. 



c. 



Periodic interviews with 
students about the learning 
expe^ences and their rela- 
tionships to the actual job 
requirements. Inquiry into 
the skills purported to be 
taught through the program 
and those actually taught, 
and skills that are needed 
but haven't been taught. 

Ratings by the evaluator or 
other outside persoh (not 
the teacher, employer, or 
student) of former- student 
performance on. the job. An 
observation instrument that 
lists job behaviors, or tasks 
should be used. 



ERIC 
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Data 'Use--Data cpllected in Phase Four^can be, used to suggest or make 
reViSions\in the objectives being taught and in the learning experiences.. 



■■ Information Desired" 

Skills required on the job, but not 
taught through the program^ 



Learning experiences that have 
been particularly effective or 
that have been ineffective. 



Methods/Means 

a. ^Direct input (interview or 

questionnaire) from employers 
and former students on skills 
not acquired through the 
training program. 

(. 

b. Direct input Obtained from 
former students during 
interviews relating their 
job experiences to the 
learning experiences. 
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Levpl of 

Use 



Level 0 
Nonuse 



Level 1 
Orientation 



Level 2 
Preparation 



Levels-of-Use Interview or Questionnaire Focal Points 



Knowledge Category 



1. Knows nothing about FACIT. 

2. May know only that Florida fias undertaken a move toward 
competency-based education. 



1. Knows what competency-based education means in general 

2. Knows that FACit is one of the state's efforts in 
helping to implement competency-based education. 

3. Knows something about the development of TACIT. 

4. Is familiar with the components of FACIT. 

5. Knows which components . (skil Is areas) are most 
important for hirr, or her. 



Knows what may be involved in implementing TACIT: 
a. Knows the sequence for implementing FACIT components. 
Knows the time frame for implementing individual ,. 
components. ' 

Knows the resources required to implement the various 
components . 

Knows the. skills that" need to be acquired. 
Knows the managerial tasks that will have to he 
carried out. 

Knows what cooperation is needed from other personnel 



b. 



Knows which of the procedures used in the past can be 
continued within the framework of CBE and FACIT. 



Level 3 

Mechanical 

Use 



Knows the processes involved in the daily planning for 
a competency-based, individualized classroom in relation 
to: 

a. Objectives 

b. Learning activities 

c. Assessment of students 

d. Management and budgeting 

e. Evaluation of instruction 

f. Student-teacher noninstructional interaction 

Knows the daily routine of implementing chosen FACIT 
components in relation to: 

a. Resource requirements 

b. Student activities 

c. Managerial tasks 

d. Evaluation tasks 
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Level of 
Use 



Know1edge--Continu ed 



Level 4a 

Routine 

Use 



Level 4b 

System 

Refinement 



1. Knows how to proceed with long-term classroom planning: 

a. Knows how to set long-term student goals. 

b. Knows how to prepare, sequence, and (-efine objectives, 

c. rinows how to sequence learning activities and 
prepare individual learning pians. 

d. knows how to assess ard report competencies. • 

e. Knows how to manage instruction, including resource 
allocation. 

f. Knows how to meet administrative requirements. 

2. Knows how to schedule and conduct Instruction in a 
competency-based system. 



Level 5 
Integration 



Level 6 
Renewal 



1. 

2. 
3. 



Knows how to identify problems in the classroom 
environment caused by the use of procedures suggested 
through FACIT. 

Knows where to look for alternatives to procedures 
suggested through FACIT.' 

Knows how to identify and integrate FACIT procedures 
with previously u^ed classroom procedures or other 
procedures that show more promise. 



Knows how to coordinate the procedures of -his or her 
classroom with those of other users of FACIT and other 
CBE systems for the purpose of making the CBE system 
more effective and efficient. 

a. Knows how to articulate his or her. program with 
those of other instructors to facilitate student 
transition from lower to higher levels or across 
programs. 

b. Knows how to articulate procedures and content 
with other instructors to avoid duplication of 

. effort. 

c. Knows how to take a group approach to instructional 
planning and delivery so that use of expertise can 
be maximized and instructional delivery can be 
ofJtimized. 



1. 
2. 



Knows how to determine ineffective FACIT procedures or 
procedures detrimental to students. 
Knows ways in which FACIT procedures can be used in 
conjunction with other procedures or could be replaced 
by other systems. 
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Level of 

Use A tquiflriy Infoririat'ion Cctcgory 



Level 0 
Nonuse 


1. Takes no initiative to acquire information about FACIT 
or other CBE-implementation efforts. 

2. Reads or absorbs descriptive information about FACIT 
or other CBE-implementation efforts only when (s)he 
has come upon it incidentally. 


Level 1 
Orientation 

♦* 

• •> 


1. Takes some initiative to acquire descriptive information 
about FACIT. 

a. Attends FACIT-awareness workshop. 

b. Discusses FACIT with other users who are more 
informed. 

2. Attends FACIT-orientation workshop. 

3. Obtains materials that (s)he plans to read through 
and implement. 


Level 2 
Preparation 

• 


1. Attends any in-depth workshops dealing with the- 
individual cofrtponent(s) that (s)he has chosen. 

a. Obtains any available supplemental materia^*? 
concerning the component(s) that (s)he wiU 
be working through. 

b. Obtains products that would result from above- 
mentioned workshops. 

2. Receives information from other FACIT users concerning 
actual requirements of implementing FACIT procedures. 


Level 3 

Mechanical 

Use 


Obtains information concerning the conditions that are 
necessary for managing the implementation of FACIT: 

a. ' Availability of materials and supplies or of 

funds to acquire materials and supplies 

b. Availability of teacher planning time 

c. Availability of administrative support 


Level 4a 

Routine 

Use 


Makes no special effort to seek additional information 
as part of his or her ongoing use of FACIT. 


• 
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Level of 

Use Acquiring Infonnation--Cont1nued 



Level 4b 

System 

Refinement 

• 


1. Obtains information on how FACIT procedures could be 
made more efficient. This could be done by-- 

a. Examining use of resources and naterials; 

b. Discussing with other users the. possibility of 
sharing learning experiences, "jeacher expertise, 
and other resources; 

c. Examining other teachers' use of similar FACIT 
concepts; and 

d. Comparing FACIT with traditional methods for the 
^, • purpose of deciding which is more efficient. 

c. Obtains 1 ntormat.! on on now rMLii proceaures couia ue 
made more effective. This could be done by-- 

a. Examining student progress files of own students, 

b. Discussing student progress with other teachers 
who use FACIT, and . 

c. Discussing new procedures with students for the 
purpose of determining whether students find these 

' proccQurcS easy to unacrbtafia ana to u^c. ^ 


Level 5 
Integration 


Obtains information on how FACIT procedures may be effec- 
tively combined with old procedures. 

A Rpriflp? which FACIT and which' new orocedures work 
best alone. 

b. Decides which procedures work best when combined. 


Level 6 
Renewal 


Obtains information about other CBE-implementation 
procedures that could be used to sipplement FACIT 
procedures or to replace ineffective FACIT procedures. 



0 
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Use 



Sharing Cate gory 



Level 0 
Nonuse 



Level 1 
Orientation 



Does not cornlnunicate with others about FACIT beyond 
possibly acknowledging that FACIT exists. 



Discusses FACIT with others in general terms as to the 
foil (wing: 
. . a. Development of FACIT 

b. Its components 

c. The appropriateness of FACIT procedures in the 
classroom 

d. The appropriate sequence of FACIT components 

e. A rationale (intended consequences) for implementation 



Level 2 
Preparation 



With other beginning and current FACIT users, discusses 
the following: 

a. Necessary resources \ 

b. ^Necessary activities 

c. Possible strategies for dealing with constraints 
(time, resources, etc.) 

Shares with other potential users any solutions to 
constraints and considers solutions th^t others may 
have to offer. ^ 



Level 3 


1. 


Discusses with other FACIT users management and.v 


Mechanical 




logistical procedures and difficulties encountered 


Use 




with FACIT. 




2. 


Shares resources and materials used to reduce 






management probl ems . 




3.. 


Shares with others any possible solutions to managerial 






and/or logistical difficulties. • ^ 

t 


Level 4a • - 


. Discusses current use of FACIT without concern for 


Routine 


making modifications. ' 


Use 






Level 4b 


1. 


Discusses the ways (s)he has integrated old procedures 


System 




with FACIT to produce a more effective and efficient 


Refinement 




learning program. 




2. 


Exchanges ideas, curriculum plans, and. materials with 






other FACIT users. 



Level 5 
Integration 



Exchanges ideas and products with other teachers in order 
to improve his or own system. The teacher's primary 
intention is to better meet the needs of students by 
modifying use of FACIT. 



-4- 



Level of 
Use 



Level 6 
Renewal 



2. 



3; 



Sharing--Continued 



Discusses and evaluates ideas and materials gained 

from sources other than FACIT. 

Discusses with others the possibility of adopting 

another system that seems more appropriate than 

FACIT or of integrating with FACIT other systems 

that are likely to enhance FACIT. 

Discusses with others the possibility of a joint 

effort for revising the FACIT system or developing 

a new system more appropriate than FACIT. 
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Level of 

Use Assessing Category 


Level 0 
Nonuse 


1. Has not assessed his or her own program to determine 
whether it shows elements of competency-based programs. 

2. Has not assessed the program to determine whether 
FACIT competencies are already used. 


Level 1 
Orientation 


Analyzes current program in order to decide whether or 
not to use FACIT. 

a. Determines the extent to which his or her program 
is competency-based. 

b. Identifies characteristics of his or her program 
that are present in FACIT. 

c. Identifies characteristics of FACIT that are not 
found in his or her. program. 


J Level 2 
Preparation 


Prepares for first use of FACIT by analyzing current 
program and detailed requirements for use of FACIT. 

a. Analyzes each component to determine whether it 
will be useful in his or her program. 

b. Analyzes, each component to determine whether the 
procedures w:ill be practical to u^e in nis or' her 
educational environment. 

c. Analyzes each competency as to estimate^ time, 
. resource, and student requirements. 

d. Uses above analysis to further include or eliminate 
components and/or competencies. 


Level 3 

Mechanical 

Use 


Assesses the FACIT system as it is used by him, or her on 
the basis, of the following: 

a. Time used to plan and collect materials required 
for elach skill or knowledge being taught 

b. Student learning time 

c. Resources (material and human) needed 

d. Impact on students 

(1) Efficiency in classroom. management 
(.2 ).• Efficiency and effectiveness in program 
planning and delivery 


Level 4a 

Routine 

Use 


1. Assessed a total competency or group of similar, skills 
as the competency or group is completed. 

a. Analyzes pretest and posttest information. * 

b. Summarizes information on student behaviors and 
comments during work periods. 

c. Decides whether the time needed to maintain 
recommended records is worthwhile. 

2. Evaluates the new teacher role as classroom manager. 

a. Determines whether students accept this new teacher 
role. 

b. Determines whether the teacher feels comfortable, 
with this new role. 



Level of 
Use 



Level 4b 

System 

Refinement 



Level 5 
Integration 



Level 6 
Renewal 



As s e s s i ng - - Con t i n ued 



Evaluates FACIT as to efficiency and effectiveness in 
order to improve its impact on students. Uses student 
performance records and their comments to determine ways 
to improve use of FACIT. 



1. Determines how a new, more effective curriculum can be 
devised using a combination of FACIT and new alternati^s 

2. Compares personal use of FACIT in the classroom with 
that of ^iher teachers. 

3. Examines Iny procedures that appear to be more efficient 
than his or her own. 

4. Decides to integrate .any procedures that are more 
efficient or effective than his or her own. 



1. Examines and evaluates any new CBErimplementation efforts 
about which information is available. 

2. Decides whether these procedures are likely to prove 
mora or less effective than those already In use in, 
his or her classroom. 

3. Decides to adopt any procedures that prove more 
effective than those already in use jn the classroom. . 
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Level, of 
Use 

Level 0 
Nonuse 



Levi 1 
Orientation 



Level 2 
.Preparation 



Level 3 

Mechanical 

Use 



Level 4a 

Routine 

Use 



Level 4b 
System 
Refinement 



Level 5 
Integration 



_ Planning Category 



Makes no specific plans to obtain information about or 
to use FACIT. 



Plans to acquire written materials explaining FACIT or 
to attend a FACIT-orientation session in order to decide 
whether or not to try using FACIT. 



1. Decides which FACIT competencies are most relevant to 
his or her immediate needs. 

2. Determines whether the instructional materials and 
other resources necessary for beginning FACIT use are 
available. 



Plans to incorporate FACIT concepts and/or procedures into 
his or „ her curriculum. 

a. Develops student learning pVans centering around 
FACIT concepts and procedures. , ^ . 

b. Makes arrang'ements to obtain necessary instructional 
materials and supplies. 

, c. PJans with other 'teachers to share resources and 
materials when possible. 



Coljects materials and information needed to Implement the 
FAC'IT approach for the entire curriculum, 



Plans changes in management and/or logistical aspects of 
FACIT implementation in order to more effectively meet 
student needs. 

a. Makes any necessary substitutions in materials or 
resources . 

. • b. Plans alternative learning experiences for students, 
c. Changes FACIT procedures that prove impractical. 



1. Plans to refine curriculum by combining most effective 
FACIT procedures with most effective procedures formerly 
used. 

2. Plans to review the curricula of other instructors who 
are implementing FACIT. 

3. Plans to incorporate any concepts or procedures (obtained 
from other instructors) that prove more efficient and 
effective than his or her own. 

4. Plans to work with other teachers in sharing areas of 
expertise, in team teaching, in presenting demonstrations, 
and in obtaining resources. 
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1 



Level of 
Use 



Level 6 
Renewal 



Planm'ng- rC ontinued 



1. Plans to review other CBE- implementation efforts. 

2. Plans to incorporate any o^ these new procedures into 
his or her curriculum and to supplement and/or replace 

' any old or FACIT procedures that have proven iry(5.dequate. 
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Level of 
Use 



Status Reporting Category 



Level 0 
Nonuse 



1; Reports that (s)he has no knowledge of FACIT. 

2. Reports little or no personal Involvement with FACIT 



tevef 1 
Orientation 



1. 

2. 

3. 

4. 

5. 
6. 



7. 



Reports that (s)he has attended a FA^IT-awareqess 

session. ' ^• 

Reports that (s)he believes at least part of th^ FACIT 

materials will be useful to him or her. 

Reports that (s)he has decided which component^ will 

be most relevant toJifs or her needs. 

Reports that (s)he has attended an in-depth workshop 

dealing with one or more of his or her chosen components 

Reports that (s)he is working through the materials. 

Reports that ('-)he has been able to master all of the 

posttests in his or her chosen components. 

Reports that (s)he is planning to incorporate pACIT 

procedures into his or her curriculum. - ^ 



Level 2 . 
Preparation 



1. Reports on how (s)he is p,lanning to use FACIT procedures 
in the classroom. 

2. Reports on efforts undertaken toward •implementing each 
' cdnponent. V; , ■ ■ 



Level 3 
Mechanical 

Use \ 



— ^» Y — ^ 

Level '4a 
Routing 

Use. \ 



Level 4b. 

System 

Refinement 



1. Rfeports on how well FACIT procedures are working and 
being received by students. 

2. Reports on how helpful the guidelines given in FACIT 
-^^er-e:4n.-tbe:jnaJiagemenJ/0^ 



Reports few, if any , problems in implementing FACIT 



■Reports that FACIT procedures are being adapted so that 
thiey will work more efficiently and effectively. 



Level 5 
Integration 



1. Reports that (s)he intends to adopt any procedures 
that in other classrooms have proven more efficient 
than some 'FACIT procedures. 

2. Reports that (s)he is involved in a team-teaching or 
sharing effort to produce a niore effective curriculum. 

3. Reports that (s)he is examining other CBE-implementation 
efforts to determine whether' they will complement FACIT. 



Level 6 
Renewal 



1. Reports that certain elements of the above-mentioned 
efforts have been integrated into his or her curriculum 
to supplement or Xpldcd any. ineffet<tive FACII«pirocedures 

2. Is ■ 



considering fnaj y ^^hanges^^in . the ^ pcit^ prj^jledu res . 



Level of 
Use 



Level 0 
Nonus^ 



1. 
2. 
3. 
4. 



Level 1 
Orientation 



Level 2 
Preparation 



1. 

2; 
1 

4. 



Level 3 
Mechanical 

Use ■ 



1. 



Performi ng Category 



Is not using FACIT or any of its components.^' 
Is not consciously using any CBE techniques.' 
Does not actively search for CBE materials. 
Takes no discernable action toward learning about FACIT. 



Gathers information about CBE -and FACIT. 

a. Reads vocational journal articles relating to CBE 
and FACIT. 

b. Attends in-service workshops dealing with CBE and 
FACIT. 

c. Talks with other teachers and with ^administrators 
about CBE- and FACIT-related concerns. 

Attends FACIT-awareness workshop. 

a. Learns about CBE concepts and techniques. 

b. Learns about FACIT and its componervts. 

G. Learns about the procedures irjvdlved in working 
through FACiT. * ' " • / 



Inspects the FACIT sysfem and decides which components 
are not relevant for him on her. \, 
Obtains necessary FACIT materials. 
Works through chosen FACIT components, takes pretests 
anrf posttests, and peirforms suggested activi^ie^. 
Makes specific plains for how the implementation of 
CBE vyi 11 proceed in his or her classroom. 



Begins to implement competency-based education in his 
or her program, using procedures and techniques 
suggested through FAC 17 - 

a. Develops or revises objectives .< 

b. Develops/or revises student learning activities. 

c. Develop^, revises, or obtains criterion-referenced 
tests. ^-^ 

d. Develops a classroom management system to meet the 
requirements of competency-based, individualized 
instruction. 

e. Plans how (s) he will evaluate instruction. 



2. Revises procedures as problems occur. 
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Level of ^ , 

•Use Perforniing--Co nt1nued 



Level 4a 

Ro'utine 

Use 

ft 

i 


0 

1. Uses all components suggestecf th^'ough FACIT.' 

a! Sets short- and long-term. gbaU with students. 

b. Selects and sequences objectives for individual 
students^. *^ 

c. ' Uses criterion-f'cfeV'enced tests. ' 

d. Provides indi vidua! fzfed learning experiences. 

e. Evaluates' instruction. 

2. Continues development of instruction to provide students 
.with, a variety of -learning experiences. 

t 

3. Manages classroom ^nd instruction eff iciently'according 
to FACIT-suggeUed procedures. " '/ 


Level 4b 

System 

Refinement 

1 


i; Revises learning experiences based on evaluative input. 
2. incorporates' into the FACIT system successful classroom 
'techniques used previously. 
' 3. Incorporates into the FACIT systm innov'al.ive techniques 
and curriculum materials. 
4. Alters. FACIT techniques to make them more suited to 
. his or her pai»ticular classroom si tu^t'ti on. 


• V 

Level 5 ^ 
Integration 


1. " Plans with other teachers an articulation of techniques 

cind learning experiences used in the various classrooms. 

2. Adapts his or her own curriculum to provide for better 
' articulation .^etween programs. 

3. Adopts techniques and procedures used by other teachers 
to make his or her classroom procedures more effective 
and efficient. 

: ' . ■ ♦ 


Level 6 
Renewal 

, v. 


r ... 

1. Investigates other CBE aftproaches. 

2. Selects qther CBE approaches to use in, addition to or 
. instead Df FACIT approaches. . 

3.. Revises the classroom procedures , using more innovative 
and effective systems. 
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